4 ms·

Most of the 'problems' there are Rust programs not micro-optimised to the ridiculous extent of the C ones, e.g. the C ones almost all use SIMD instrinsics or fe

by dbaupp 12y ago

Most of the 'problems' there are Rust programs not micro-optimised to the ridiculous extent of the C ones, e.g. the C ones almost all use SIMD instrinsics or features like OpenMP. Rust is a new, experimental language and the support for SIMD is still in development, and we still don't have much in the way of data-parallel libraries. The type system has been expressly adjusted to allow for safe data-parallel libraries though, so they will appear. Also, a few other points: - Rust allows for opting in to unsafe code (with rules and overhead similar to C), which can be used for performance optimisations. Last time I checked, none of the Rust benchmark programs used any unsafe. (On the other hand, all of the C programs are effectively using `unsafe` code always) - C is using highly optimised libraries; Rust has zero overhead FFI, and so can easily call into them, but is not for those benchmark (e.g. the pidigits one is just a benchmark of "are you using GMP?", if the Rust was written to use it, there is no reason it would be slower than C) - The worst offender (reverse-complement) is apparently written without any thought of performance, e.g. it is doing formatted IO for each and every character it emits, rather than writing to an in-memory buffer and dumping that in one go like the C. In summary, those benchmarks do not reflect the fundamental speeds of the languages, other than the suboptimal SIMD support in Rust.

acqq 12y ago
Improve the benchmarks and submit them to the shutout. Use the benchmarks during the development too to remain focused on the performace. It's not helpful when somebody in the small circle knows that the benchmark can be implemented better if there aren't any reference examples. I believe Servo is your reference, but you definitely need to have a bit wider horizons than that. The small benchmarks are good representation of the rest of the world. And the good advertisement, once they are fast. Please don't ignore them.
- dbaupp 12y ago
  They are not being ignored. Better versions of those benchmarks exist in-tree: https://github.com/rust-lang/rust/tree/master/src/test/bench https://github.com/rust-lang/rust/tree/master/src/test/bench (the shootout-....rs). The only reason they're not on the website is licensing problems (which have recently been resolved) and the fact that the benchmarks game is using the 0.11 release (while the in-tree benchmarks are kept up-to-date with the master branch and so don't necessarily compile with 0.11); this new release represents a good chance to push them up. Those small benchmarks are not good representations of the rest of the world; they are contrived and limited problems, with some fairly arbitrary rules about which languages/implementations are valid to include, e.g. PyPy is not allowed, and Java gets JIT warm-up time etc.
  - TheLoneWolfling 12y ago
    Yow. I didn't know that. (Seems like a better way to test would be to allow anything, with three sets of numbers. One for "time from source to as far as you can go without input" (i.e. compilation time, loading the source into RAM, that sort of thing) "time from input to end of first run", one for "time for nth run", with n being high enough that the timing settles. So after any JITters are done, that sort of thing.)
    - igouy 12y ago
      You "didn't know that" because it isn't true.
  - igouy 12y ago
    > The only reason they're not on the website is… … that no one has contributed them. > … with some fairly arbitrary rules about which languages/implementations are valid to include, e.g. PyPy is not allowed … Hundreds of programming language implementations are not included! It would take more time than I choose to donate. Been there; done that. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex > … and Java gets JIT warm-up time etc. Java does not get JIT warm-up time! Please stop making up misinformation! http://benchmarksgame.alioth.debian.org/play.html#measure http://benchmarksgame.alioth.debian.org/play.html#measure http://benchmarksgame.alioth.debian.org/play.html#java http://benchmarksgame.alioth.debian.org/play.html#java
    - dbaupp 12y ago
      The site could be renamed to be "The Computer Language Implementation Benchmarks Game", since it's not testing the language speed (best approximated by the fastest known implementation), just certain implementations, some of which are designed with priorities above speed in mind (e.g. cPython). > Java does not get JIT warm-up time! Please stop making up misinformation! Oh, sorry! I must've been misremembering something someone told me. (Although, how does `Cold` differ from `Usual` there? It's not clear from the text what the difference is.)
dbaupp 12y ago
In fact: I translated the C pidigits program to Rust: https://gist.github.com/huonw/4b326e9a73a40df664cd https://gist.github.com/huonw/4b326e9a73a40df664cd All it takes is someone to write a wrapper around gmp providing those parts of the API and it will be less verbose and not require the `unsafe` everywhere. Timing: $ /usr/bin/gcc -pipe -Wall -O3 -fomit-frame-pointer -march=native pidigits.c -o pidigits.gcc_run -lgmp $ time ./pidigits.gcc_run 10000 > /dev/null real 0m0.980s user 0m0.976s sys 0m0.000s $ rustc --opt-level=3 pidigits.rs -o pidigits.rust $ time ./pidigits.rust 10000 > /dev/null real 0m0.987s user 0m0.984s sys 0m0.000s
- acqq 12y ago
  Bravo! It's still a good and important demonstration of what the language features. Gmp is LGPL, so you should add the real safe wrappers (to avoid doing everything unsafe) and allow dynamic linking. Can Rust link dynamically?
  - dbaupp 12y ago
    Yes, it can link dynamically: $ ldd pidigits.rust linux-vdso.so.1 (0x00007fff8b9e7000) libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f5c233be000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5c231ba000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f5c22f9c000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f5c22d86000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5c229de000) /lib64/ld-linux-x86-64.so.2 (0x00007f5c2372b000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5c226dc000)
    - acqq 12y ago
      You see, I'm certainly not the only one who wouldn't know all this unless there are good benchmark examples (or have the luck of being able to ask and receive the answer from you. I believe Go couldn't do that once, I don't know the current state. Thank you. But a lot of people wouldn't get so far. Therefore please do promote good examples of the fast code that do something real, like these benchmarks.

by dbaupp 12y ago

acqq 12y ago

Improve the benchmarks and submit them to the shutout. Use the benchmarks during the development too to remain focused on the performace. It's not helpful when somebody in the small circle knows that the benchmark can be implemented better if there aren't any reference examples. I believe Servo is your reference, but you definitely need to have a bit wider horizons than that. The small benchmarks are good representation of the rest of the world. And the good advertisement, once they are fast. Please don't ignore them.

dbaupp 12y ago
They are not being ignored. Better versions of those benchmarks exist in-tree: https://github.com/rust-lang/rust/tree/master/src/test/bench https://github.com/rust-lang/rust/tree/master/src/test/bench (the shootout-....rs). The only reason they're not on the website is licensing problems (which have recently been resolved) and the fact that the benchmarks game is using the 0.11 release (while the in-tree benchmarks are kept up-to-date with the master branch and so don't necessarily compile with 0.11); this new release represents a good chance to push them up. Those small benchmarks are not good representations of the rest of the world; they are contrived and limited problems, with some fairly arbitrary rules about which languages/implementations are valid to include, e.g. PyPy is not allowed, and Java gets JIT warm-up time etc.
- TheLoneWolfling 12y ago
  Yow. I didn't know that. (Seems like a better way to test would be to allow anything, with three sets of numbers. One for "time from source to as far as you can go without input" (i.e. compilation time, loading the source into RAM, that sort of thing) "time from input to end of first run", one for "time for nth run", with n being high enough that the timing settles. So after any JITters are done, that sort of thing.)
  - igouy 12y ago
    You "didn't know that" because it isn't true.
- igouy 12y ago
  > The only reason they're not on the website is… … that no one has contributed them. > … with some fairly arbitrary rules about which languages/implementations are valid to include, e.g. PyPy is not allowed … Hundreds of programming language implementations are not included! It would take more time than I choose to donate. Been there; done that. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex > … and Java gets JIT warm-up time etc. Java does not get JIT warm-up time! Please stop making up misinformation! http://benchmarksgame.alioth.debian.org/play.html#measure http://benchmarksgame.alioth.debian.org/play.html#measure http://benchmarksgame.alioth.debian.org/play.html#java http://benchmarksgame.alioth.debian.org/play.html#java
  - dbaupp 12y ago
    The site could be renamed to be "The Computer Language Implementation Benchmarks Game", since it's not testing the language speed (best approximated by the fastest known implementation), just certain implementations, some of which are designed with priorities above speed in mind (e.g. cPython). > Java does not get JIT warm-up time! Please stop making up misinformation! Oh, sorry! I must've been misremembering something someone told me. (Although, how does `Cold` differ from `Usual` there? It's not clear from the text what the difference is.)

dbaupp 12y ago

In fact: I translated the C pidigits program to Rust: https://gist.github.com/huonw/4b326e9a73a40df664cd https://gist.github.com/huonw/4b326e9a73a40df664cd All it takes is someone to write a wrapper around gmp providing those parts of the API and it will be less verbose and not require the `unsafe` everywhere. Timing: $ /usr/bin/gcc -pipe -Wall -O3 -fomit-frame-pointer -march=native pidigits.c -o pidigits.gcc_run -lgmp $ time ./pidigits.gcc_run 10000 > /dev/null real 0m0.980s user 0m0.976s sys 0m0.000s $ rustc --opt-level=3 pidigits.rs -o pidigits.rust $ time ./pidigits.rust 10000 > /dev/null real 0m0.987s user 0m0.984s sys 0m0.000s

acqq 12y ago
Bravo! It's still a good and important demonstration of what the language features. Gmp is LGPL, so you should add the real safe wrappers (to avoid doing everything unsafe) and allow dynamic linking. Can Rust link dynamically?
- dbaupp 12y ago
  Yes, it can link dynamically: $ ldd pidigits.rust linux-vdso.so.1 (0x00007fff8b9e7000) libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f5c233be000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5c231ba000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f5c22f9c000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f5c22d86000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5c229de000) /lib64/ld-linux-x86-64.so.2 (0x00007f5c2372b000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5c226dc000)
  - acqq 12y ago
    You see, I'm certainly not the only one who wouldn't know all this unless there are good benchmark examples (or have the luck of being able to ask and receive the answer from you. I believe Go couldn't do that once, I don't know the current state. Thank you. But a lot of people wouldn't get so far. Therefore please do promote good examples of the fast code that do something real, like these benchmarks.