3 ms·

Different languages' benchmarks might not be equally well-written / optimized. In particular, I'd expect C and Rust to be very close to each other, and a 20% ga

Different languages' benchmarks might not be equally well-written / optimized. In particular, I'd expect C and Rust to be very close to each other, and a 20% gap between them is a red flag. Rules like "code should be simple, as in, easy to read and understand" are also hard to judge, especially near the top of the list where there's a lot of pressure to optimize. Is SIMD easy to understand? What if it's in a library? What if the library was written specifically for this benchmark? Etc. I think https://benchmarksgame-team.pages.debian.net/benchmarksgame/ https://benchmarksgame-team.pages.debian.net/benchmarksgame/ has to deal with every possible permutation of this debate.

remcob 8y ago
Not necessarily. Because C allows the pointer manipulation, the compiler can in general not make assumptions about pointer aliasing. This prevents some optimizations. In Rust, the compiler has more information/control over memory layout/lifetime and can therefore make stronger optimizations. Automatic vectorization is an area where this helps a lot, and raytracing can benefit a lot here. 20% sounds reasonable to me.
- oconnor663 8y ago
  That's a good point, I didn't think about the effect on autovectorization. Do you think that's what's happening here? My impression was that getting good vector code out of the compiler usually requires manually tuning things.
- cesarb 8y ago
  > pointer aliasing Unfortunately, due to LLVM bugs, the Rust developers had to disable that optimization, more than once. I don't know whether the "1.13.0-nightly" he used has that optimization enabled or disabled. (See https://github.com/rust-lang/rust/issues/31681 https://github.com/rust-lang/rust/issues/31681 and https://github.com/rust-lang/rust/issues/54878 https://github.com/rust-lang/rust/issues/54878 for the relevant Rust issues.)
- bjourne 8y ago
  Well, generally, perhaps. But any performance oriented C programmer worth his or her salt would be aware of aliasing issues and write code in such a way that it doesn't cause problems for the compiler. Plus, this is a toy benchmark of a few hundred lines so the compiler can do full-program analysis. So the 20% difference is indeed a smell. Looking at the crb*.c files, structs are passed as pointers and not by value. This makes it harder for the compiler to analyze the data flow which I would bet is part of the reason Rust is faster here.

remcob 8y ago

Not necessarily. Because C allows the pointer manipulation, the compiler can in general not make assumptions about pointer aliasing. This prevents some optimizations. In Rust, the compiler has more information/control over memory layout/lifetime and can therefore make stronger optimizations. Automatic vectorization is an area where this helps a lot, and raytracing can benefit a lot here. 20% sounds reasonable to me.

oconnor663 8y ago
That's a good point, I didn't think about the effect on autovectorization. Do you think that's what's happening here? My impression was that getting good vector code out of the compiler usually requires manually tuning things.
cesarb 8y ago
> pointer aliasing Unfortunately, due to LLVM bugs, the Rust developers had to disable that optimization, more than once. I don't know whether the "1.13.0-nightly" he used has that optimization enabled or disabled. (See https://github.com/rust-lang/rust/issues/31681 https://github.com/rust-lang/rust/issues/31681 and https://github.com/rust-lang/rust/issues/54878 https://github.com/rust-lang/rust/issues/54878 for the relevant Rust issues.)
bjourne 8y ago
Well, generally, perhaps. But any performance oriented C programmer worth his or her salt would be aware of aliasing issues and write code in such a way that it doesn't cause problems for the compiler. Plus, this is a toy benchmark of a few hundred lines so the compiler can do full-program analysis. So the 20% difference is indeed a smell. Looking at the crb*.c files, structs are passed as pointers and not by value. This makes it harder for the compiler to analyze the data flow which I would bet is part of the reason Rust is faster here.