10 ms·

This is a wonderfully written comparison benchmark and it deserves attention even for that reason alone. It knows its target audience, explains what's going on

This is a wonderfully written comparison benchmark and it deserves attention even for that reason alone. It knows its target audience, explains what's going on succinctly but completely, and avoids most of the usual benchmark pitfalls that result in comparing apples to oranges. Great job. The one glaring issue that is ignored is the floating point model used. I understand Rust still doesn't have a usable equivalent to -ffast-math, so I assume it wasn't used for that reason. Some discussion of whether it's permissible in this algorithm (I believe so?) and how much advantage that might give to C++ seems crucial when performance is a priority. Ironically, reading this has further convinced me that, for all of its disadvantages, I much prefer C++ to Rust for my needs. I'm sure others will draw the opposite conclusion and that's great. Rust is a language that clearly knows what it wants and if your priorities are aligned with that, the performance gap is shrinking and implementation-related reasons to avoid it are rapidly decreasing in number.

mratsim 7y ago
ffast-math is not needed when you unroll manually and write your own accumulators. See my benchmarks [1] with fast-math in the middle and the generated assembly comparison at the bottom. The base language is Nim but pure C code should generate the same assembly. [1] https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1187864323d85527d18c/benchmarks/fp_reduction_latency/reduction_bench.nim#L304-L331 https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1....
- keldaris 7y ago
  That's absolutely right (and thank you for the reference), but this benchmark starts off with fairly naive code where -ffast-math may still help considerably.
  - mratsim 7y ago
    If in-between the start and the end, I went up to 8 accumulators and in files in the same folder I have the same with SSE and AVX instructions. I actually wanted to know exactly what ffast-math did, down to the codegen (so you have assembly at the bottom).
dgellow 7y ago
> reading this has further convinced me that, for all of its disadvantages, I much prefer C++ to Rust for my needs. What are your needs, if I may ask? I'm not asking this to start a thread "rust/C++ is better", I'm just curious regarding situations in which someone decides that one language matches their needs more than another, it's always interesting to see what boundaries people consider for those decisions.
- keldaris 7y ago
  I mostly use C++ for numerical simulations in physics and associated code (analysis, some visualization, etc.). That means my primary consideration is the ease and convenience of writing high performance code for a narrow set of hardware. I care about the quality of tooling, especially for performance analysis (including things like likwid [1]) and GPGPU computation. I do not care about safety (memory or otherwise) - my code doesn't take arbitrary input, run on shared hardware, do much of anything over networks or have memory safety crashes. From this rather narrow point of view, Rust does very little to help and quite a lot to hinder me. Rust is very much about memory safety - an issue extremely far down my list of concerns - and to me the borrow checker is an anti-feature I'd love to turn off. None of this is in any way an indictment of Rust - it looks like a very well designed language that knows what it wants to accomplish. It just happens to want the exact opposite things from what I want, and that's fine. I know I'm in a weird minority (most of the people who think like I do seem to be game engine developers). [1] https://github.com/RRZE-HPC/likwid https://github.com/RRZE-HPC/likwid
  - red75prime 7y ago
    How are you dealing with data races? Manual synchronization (memory barriers, and the like)?
    - keldaris 7y ago
      For the vast majority of cases I've found it possible (and very much worthwhile) to just think a bit harder and avoid race conditions by construction (for instance, reformulating an algorithm so that writes only occur to different cache lines in a single pass, etc.). That's obviously not always feasible, but in my experience people give up far too quickly. For the remaining few cases - yes, manually inserting the minimal necessary amount of synchronization (on x86-64 that's often a single instruction). On GPUs you obviously need memory barriers fairly frequently, but that's much less of an issue than on CPUs. Anyway, I don't have any faith in "smart" compilers or language features (which have large costs elsewhere) that try to free me from the simple necessity to carefully think through what my algorithm actually does before I implement it. If you care about getting close to optimal performance, you'll have to do the thinking anyway.
pcwalton 7y ago
Honestly, a lot of this comes down to whether you prefer iterators or C-style for loops. I prefer iterators myself, as it's way too easy to make C-style for loops unreadable. Reasonable people can disagree, of course.
nwallin 7y ago
-ffast-math quickly loses its value once you start dispatching raw intrinsics. -ffast-math tells the compiler "just figure or the math for me, I know what I want but don't know how to get it", but raw intrinsics say "use this instruction, but deal with manual register management for me." If you take the code `v /= v.length();` and given -ffast-math the compiler can easily compile that as a mul, two fmas, an rsqrt, and three muls. But if you take the code: (can't format right on mobile) (edit: fixed on desktop) __m256 len = _mm256_mul_ps(v.x, v.x); len = _mm256_add_ps(len, _mm256_mul_ps(v.y,v.y)); len = _mm256_add_ps(len, _mm256_mul(v.z,v.z)); len = _mm256_sqrt_ps(len); v.x = _mm256_div_ps(v.x, len); v.y = _mm256_div_ps(v.y, len); v.z = _mm256_div_ps(v.z, len); You've told the compiler exactly which instructions to execute. And it will do so faithfully even though it's like two orders of magnitude slower. I would be very interested to see -Ofast used before SIMD intrinsics start getting thrown around, but it has no value afterwards. The code in the linked article is eventually almost all SIMD intrinsics (v4?) so it won't make a difference.
- mkbosmans 7y ago
  What are you talking about? Intrinsics don't force the compiler to use a specific instruction. It's not even a 1:1 mapping. I think all compilers will fuse separate add and mul instrinsics to fmadd. Both GCC and Clang will convert the _mm256_div_ps to a vmulps. GCC calculates the scaling factor by combining vsqrtps and vrcpps. Clang will emit a vrsqrtps instruction, but as that has only 12 bits of guaranteed accuracy, it fixes the result up before using it to scale x, y and z. https://godbolt.org/z/rG2rH8 https://godbolt.org/z/rG2rH8

mratsim 7y ago

ffast-math is not needed when you unroll manually and write your own accumulators. See my benchmarks [1] with fast-math in the middle and the generated assembly comparison at the bottom. The base language is Nim but pure C code should generate the same assembly. [1] https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1187864323d85527d18c/benchmarks/fp_reduction_latency/reduction_bench.nim#L304-L331 https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1....

keldaris 7y ago
That's absolutely right (and thank you for the reference), but this benchmark starts off with fairly naive code where -ffast-math may still help considerably.
- mratsim 7y ago
  If in-between the start and the end, I went up to 8 accumulators and in files in the same folder I have the same with SSE and AVX instructions. I actually wanted to know exactly what ffast-math did, down to the codegen (so you have assembly at the bottom).

dgellow 7y ago

> reading this has further convinced me that, for all of its disadvantages, I much prefer C++ to Rust for my needs. What are your needs, if I may ask? I'm not asking this to start a thread "rust/C++ is better", I'm just curious regarding situations in which someone decides that one language matches their needs more than another, it's always interesting to see what boundaries people consider for those decisions.

keldaris 7y ago
I mostly use C++ for numerical simulations in physics and associated code (analysis, some visualization, etc.). That means my primary consideration is the ease and convenience of writing high performance code for a narrow set of hardware. I care about the quality of tooling, especially for performance analysis (including things like likwid [1]) and GPGPU computation. I do not care about safety (memory or otherwise) - my code doesn't take arbitrary input, run on shared hardware, do much of anything over networks or have memory safety crashes. From this rather narrow point of view, Rust does very little to help and quite a lot to hinder me. Rust is very much about memory safety - an issue extremely far down my list of concerns - and to me the borrow checker is an anti-feature I'd love to turn off. None of this is in any way an indictment of Rust - it looks like a very well designed language that knows what it wants to accomplish. It just happens to want the exact opposite things from what I want, and that's fine. I know I'm in a weird minority (most of the people who think like I do seem to be game engine developers). [1] https://github.com/RRZE-HPC/likwid https://github.com/RRZE-HPC/likwid
- red75prime 7y ago
  How are you dealing with data races? Manual synchronization (memory barriers, and the like)?
  - keldaris 7y ago
    For the vast majority of cases I've found it possible (and very much worthwhile) to just think a bit harder and avoid race conditions by construction (for instance, reformulating an algorithm so that writes only occur to different cache lines in a single pass, etc.). That's obviously not always feasible, but in my experience people give up far too quickly. For the remaining few cases - yes, manually inserting the minimal necessary amount of synchronization (on x86-64 that's often a single instruction). On GPUs you obviously need memory barriers fairly frequently, but that's much less of an issue than on CPUs. Anyway, I don't have any faith in "smart" compilers or language features (which have large costs elsewhere) that try to free me from the simple necessity to carefully think through what my algorithm actually does before I implement it. If you care about getting close to optimal performance, you'll have to do the thinking anyway.

pcwalton 7y ago

Honestly, a lot of this comes down to whether you prefer iterators or C-style for loops. I prefer iterators myself, as it's way too easy to make C-style for loops unreadable. Reasonable people can disagree, of course.

nwallin 7y ago

-ffast-math quickly loses its value once you start dispatching raw intrinsics. -ffast-math tells the compiler "just figure or the math for me, I know what I want but don't know how to get it", but raw intrinsics say "use this instruction, but deal with manual register management for me." If you take the code `v /= v.length();` and given -ffast-math the compiler can easily compile that as a mul, two fmas, an rsqrt, and three muls. But if you take the code: (can't format right on mobile) (edit: fixed on desktop) __m256 len = _mm256_mul_ps(v.x, v.x); len = _mm256_add_ps(len, _mm256_mul_ps(v.y,v.y)); len = _mm256_add_ps(len, _mm256_mul(v.z,v.z)); len = _mm256_sqrt_ps(len); v.x = _mm256_div_ps(v.x, len); v.y = _mm256_div_ps(v.y, len); v.z = _mm256_div_ps(v.z, len); You've told the compiler exactly which instructions to execute. And it will do so faithfully even though it's like two orders of magnitude slower. I would be very interested to see -Ofast used before SIMD intrinsics start getting thrown around, but it has no value afterwards. The code in the linked article is eventually almost all SIMD intrinsics (v4?) so it won't make a difference.

mkbosmans 7y ago
What are you talking about? Intrinsics don't force the compiler to use a specific instruction. It's not even a 1:1 mapping. I think all compilers will fuse separate add and mul instrinsics to fmadd. Both GCC and Clang will convert the _mm256_div_ps to a vmulps. GCC calculates the scaling factor by combining vsqrtps and vrcpps. Clang will emit a vrsqrtps instruction, but as that has only 12 bits of guaranteed accuracy, it fixes the result up before using it to scale x, y and z. https://godbolt.org/z/rG2rH8 https://godbolt.org/z/rG2rH8