6 ms·

Rust vs. C++ Benchmark Game

51 points by tete 12y ago

farresito 12y ago
While I'm not a huge fan of benchmarks, this is quite impressive, given the age of Rust. C++ does better memory-wise in pretty much all benchmarks.
- igouy 12y ago
  Don't read too much into those memory use measurements - they are just enough to suggest programmers might have made a space/time trade-off, nothing more.
  - zokier 12y ago
    Considering that memory management is one of the headline features of Rust, memory use is certainly important factor to analyze.
    - igouy 12y ago
      That may well be, but the benchmarks game measurements are not a good basis for that analysis.
- dbaupp 12y ago
  The main problem is the large (multi-megabytes) runtime that Rust currently is forced to load by default. This is disappearing.
shmerl 12y ago
The server is down for me.
- tux3 12y ago
  Same, but google had the time to cache it. https://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Fbenchmarksgame.alioth.debian.org%2Fu64q%2Fbenchmark.php%3Ftest%3Dall%26lang%3Drust%26lang2%3Dgpp%26data%3Du64q https://webcache.googleusercontent.com/search?q=cache%3Ahttp...
- deleted 12y ago
  [deleted]
- igouy 12y ago
  And up, presumably once this discussion thread moved off the front page ;-)
sqrt17 12y ago
Rust seems to do worse on the string-y tasks (fannkuch, k-nucleotides, regex-dna). A lot if you look at single-thread performance. Does it handle strings differently from C++? (From what I understood, Rust gives you similar control over memory allocation as [a sane subset of] C++, so intuitively it should come out relatively close)
- burntsushi 12y ago
  With respect to regex-dna, there is still substantial room for improvement in the Rust library. It's currently using a full NFA simulation (albeit specialized to native Rust code with the `regex!` macro), while the competing C++ library (RE2) has been tricked out by Russ Cox with a parallel DFA implementation. :-)
pptr1 12y ago
Rust really interest me. However I want them to get to 1.0 before I can evaluate whether or not to use Rust in my app. Currently I am using Go and it stable for my needs. Rust could make me switch once they get to 1.0 and people start writing more projects in Rust. I want to add that stability in deciding to use a language in an application is very important. How good the standard library is and how much traction the language is gaining in projects and particularly open source projects is also important.
- tete 12y ago
  I agree. However sometimes it's good to do tiny side projects in a language. This allows you to learn, see if it's really nice, help the language and its ecosystem to develop and become stable. For use outside of production it seems okay. There are two Sinatra/Flask/Express style web frameworks (Nickel and Iron), a browser engine (Servo), a game engine (Piston) actively developed. So if you have interest in any of those or interest in implementing something like that you might be able to make your project shine. Also really nice on your portfolio if you ever want to get a job where Rust is already in use. Or one just has fun with it. :D
  - pptr1 12y ago
    Agreed. I would do a small project just for fun.
tete 12y ago
Found on the Rust subreddit. https://pay.reddit.com/r/rust/comments/2jm4vq/rust_0120_vs_c_computer_language_benchmarks_game/ https://pay.reddit.com/r/rust/comments/2jm4vq/rust_0120_vs_c...
- capnrefsmmat 12y ago
  Incidentally -- reddit supports https by default now, so you don't need to use pay.reddit.com anymore.
pjmlp 12y ago
Actually LLVM vs GCC backend benchmark.
- igouy 12y ago
  Is there a Rust tool chain based on GCC?
  - adrusi 12y ago
    Not with full up to date language support, but there's a mature c++ toolchain on top of llvm
jyxent 12y ago
As pointed out in the reddit thread, this may be more a measure of the llvm backend used by rust to the gcc backend. A more accurate comparison would be against clang.
- igouy 12y ago
  The Mono C# programs also use llvm ;-) A more accurate comparison? A different comparison would be against clang. The benchmarks game might well be described as "a good starting point" http://algs4.cs.princeton.edu/faq/ http://algs4.cs.princeton.edu/faq/ but let's not confuse that with some kind of definitive final analysis. (Edit: Say what you disagree with, don't just down vote.)
- acqq 12y ago
  It doesn't matter much what the backends are, it's important to be able to demonstrate the effective implementation of the problems posed by the tasks being benchmarked. Especially for the new language. Of course, if your language is designed as to have the limited chances of being universaly fast, it's another story. But Rust is designed to be able to reach at least the speeds of C and also allow for a more modern approach. That's the real worth of the contributions to the "benchmark game:" demonstrating the capabilites of the language, even if it's just on some of all of possible tasks. See also Rust vs Haskell: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=rust&lang2=ghc&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... And Rust vs Go: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=rust&lang2=go&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
- joelthelion 12y ago
  It's not that easy to implement a language that's expressive with this kind of performance, even if you use llvm. Kudos to the Rust guys!
- SwellJoe 12y ago
  I was under the impression that clang was still, generally, slightly slower than GCC for C++. Perhaps this has changed in recent years (as clang and llvm have been seeing tons of development effort and funding).
bsaul 12y ago
I've only started to investigate rust for server side programming ( websocket in particular) but saw many discussions about the limitations rust has regarding parallel programming. Yet, iirc servo did parallel rendering of html page, so that seems a contradiction. Anyone here knows about that ? Specifically, i'm currently hesitating between golang and rust for coding a small Websocket chat server. I'm ready to use beta code because rust seems more promising to me in the long term than golang, but not if the language itself has severe fundamental limitations on server side programming.
- m0th87 12y ago
  Depends on the kinds of parallelism you want. Rust has built-in support for parallelization, but tasks are mapped 1:1 to threads. Currently M:N threading is also supported, but that will be removed. Consequently, applications requiring lots of tasks (including highly-concurrent websocket servers) won't do well with the built-in mechanism. Context: https://news.ycombinator.com/item?id=8459888 https://news.ycombinator.com/item?id=8459888
  - rictic 12y ago
    Green threads (M:N) do seem to be available as a library though: http://doc.rust-lang.org/green/ http://doc.rust-lang.org/green/
    - m0th87 12y ago
      That too is being removed: https://github.com/rust-lang/rfcs/pull/230 https://github.com/rust-lang/rfcs/pull/230
      - adrusi 12y ago
        Its being removed from the standard library. As I understand it will still be maintained as a separate library.
      - joelthelion 12y ago
        It's not being removed, it's being moved from the core of the language to an external library.
general_failure 12y ago
Very impressive. What we need now is some sort of a good mainstream all that uses it to spread its popularity.. How is rust when it comes to deployment. C++ is a real pain to deploy for large projects. Compilation setup is also a massive headache in c++.
- steveklabnik 12y ago
  With Cargo, getting a nice repeatable build is much, much easier. There's still work to be done on this front, but I'm _incredibly_ excited by this RFC: https://github.com/rust-lang/rfcs/pull/403 https://github.com/rust-lang/rfcs/pull/403
steveklabnik 12y ago
This issue on the Rust repo is very relevant: https://github.com/rust-lang/rust/issues/18085 https://github.com/rust-lang/rust/issues/18085
rando289 12y ago
I wish there was a D comparison. It seems to have similar goals.
- igouy 12y ago
  Make your wish reality - take the program source code and the measurement scripts and publish your own measurements. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex If you have a linux install the benchmarks game scripts should work fine, and you'd probably get help from the D community -- http://forum.dlang.org/thread/lv7h1r$mg3$2@digitalmars.com?page=3#post-juawuohxdfcxtbraejnv:40forum.dlang.org http://forum.dlang.org/thread/lv7h1r$mg3$2@digitalmars.com?p...

51 points by tete 12y ago

While I'm not a huge fan of benchmarks, this is quite impressive, given the age of Rust. C++ does better memory-wise in pretty much all benchmarks.

igouy 12y ago
Don't read too much into those memory use measurements - they are just enough to suggest programmers might have made a space/time trade-off, nothing more.
- zokier 12y ago
  Considering that memory management is one of the headline features of Rust, memory use is certainly important factor to analyze.
  - igouy 12y ago
    That may well be, but the benchmarks game measurements are not a good basis for that analysis.
dbaupp 12y ago
The main problem is the large (multi-megabytes) runtime that Rust currently is forced to load by default. This is disappearing.

shmerl 12y ago

The server is down for me.

tux3 12y ago
Same, but google had the time to cache it. https://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Fbenchmarksgame.alioth.debian.org%2Fu64q%2Fbenchmark.php%3Ftest%3Dall%26lang%3Drust%26lang2%3Dgpp%26data%3Du64q https://webcache.googleusercontent.com/search?q=cache%3Ahttp...
deleted 12y ago
[deleted]
igouy 12y ago
And up, presumably once this discussion thread moved off the front page ;-)

sqrt17 12y ago

Rust seems to do worse on the string-y tasks (fannkuch, k-nucleotides, regex-dna). A lot if you look at single-thread performance. Does it handle strings differently from C++? (From what I understood, Rust gives you similar control over memory allocation as [a sane subset of] C++, so intuitively it should come out relatively close)

burntsushi 12y ago
With respect to regex-dna, there is still substantial room for improvement in the Rust library. It's currently using a full NFA simulation (albeit specialized to native Rust code with the `regex!` macro), while the competing C++ library (RE2) has been tricked out by Russ Cox with a parallel DFA implementation. :-)

pptr1 12y ago

Rust really interest me. However I want them to get to 1.0 before I can evaluate whether or not to use Rust in my app. Currently I am using Go and it stable for my needs. Rust could make me switch once they get to 1.0 and people start writing more projects in Rust. I want to add that stability in deciding to use a language in an application is very important. How good the standard library is and how much traction the language is gaining in projects and particularly open source projects is also important.

tete 12y ago
I agree. However sometimes it's good to do tiny side projects in a language. This allows you to learn, see if it's really nice, help the language and its ecosystem to develop and become stable. For use outside of production it seems okay. There are two Sinatra/Flask/Express style web frameworks (Nickel and Iron), a browser engine (Servo), a game engine (Piston) actively developed. So if you have interest in any of those or interest in implementing something like that you might be able to make your project shine. Also really nice on your portfolio if you ever want to get a job where Rust is already in use. Or one just has fun with it. :D
- pptr1 12y ago
  Agreed. I would do a small project just for fun.

tete 12y ago

Found on the Rust subreddit. https://pay.reddit.com/r/rust/comments/2jm4vq/rust_0120_vs_c_computer_language_benchmarks_game/ https://pay.reddit.com/r/rust/comments/2jm4vq/rust_0120_vs_c...

capnrefsmmat 12y ago
Incidentally -- reddit supports https by default now, so you don't need to use pay.reddit.com anymore.

pjmlp 12y ago

Actually LLVM vs GCC backend benchmark.

igouy 12y ago
Is there a Rust tool chain based on GCC?
- adrusi 12y ago
  Not with full up to date language support, but there's a mature c++ toolchain on top of llvm

jyxent 12y ago

As pointed out in the reddit thread, this may be more a measure of the llvm backend used by rust to the gcc backend. A more accurate comparison would be against clang.

igouy 12y ago
The Mono C# programs also use llvm ;-) A more accurate comparison? A different comparison would be against clang. The benchmarks game might well be described as "a good starting point" http://algs4.cs.princeton.edu/faq/ http://algs4.cs.princeton.edu/faq/ but let's not confuse that with some kind of definitive final analysis. (Edit: Say what you disagree with, don't just down vote.)
acqq 12y ago
It doesn't matter much what the backends are, it's important to be able to demonstrate the effective implementation of the problems posed by the tasks being benchmarked. Especially for the new language. Of course, if your language is designed as to have the limited chances of being universaly fast, it's another story. But Rust is designed to be able to reach at least the speeds of C and also allow for a more modern approach. That's the real worth of the contributions to the "benchmark game:" demonstrating the capabilites of the language, even if it's just on some of all of possible tasks. See also Rust vs Haskell: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=rust&lang2=ghc&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... And Rust vs Go: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=rust&lang2=go&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
joelthelion 12y ago
It's not that easy to implement a language that's expressive with this kind of performance, even if you use llvm. Kudos to the Rust guys!
SwellJoe 12y ago
I was under the impression that clang was still, generally, slightly slower than GCC for C++. Perhaps this has changed in recent years (as clang and llvm have been seeing tons of development effort and funding).

bsaul 12y ago

I've only started to investigate rust for server side programming ( websocket in particular) but saw many discussions about the limitations rust has regarding parallel programming. Yet, iirc servo did parallel rendering of html page, so that seems a contradiction. Anyone here knows about that ? Specifically, i'm currently hesitating between golang and rust for coding a small Websocket chat server. I'm ready to use beta code because rust seems more promising to me in the long term than golang, but not if the language itself has severe fundamental limitations on server side programming.

m0th87 12y ago
Depends on the kinds of parallelism you want. Rust has built-in support for parallelization, but tasks are mapped 1:1 to threads. Currently M:N threading is also supported, but that will be removed. Consequently, applications requiring lots of tasks (including highly-concurrent websocket servers) won't do well with the built-in mechanism. Context: https://news.ycombinator.com/item?id=8459888 https://news.ycombinator.com/item?id=8459888
- rictic 12y ago
  Green threads (M:N) do seem to be available as a library though: http://doc.rust-lang.org/green/ http://doc.rust-lang.org/green/
  - m0th87 12y ago
    That too is being removed: https://github.com/rust-lang/rfcs/pull/230 https://github.com/rust-lang/rfcs/pull/230
    - adrusi 12y ago
      Its being removed from the standard library. As I understand it will still be maintained as a separate library.
    - joelthelion 12y ago
      It's not being removed, it's being moved from the core of the language to an external library.

general_failure 12y ago

Very impressive. What we need now is some sort of a good mainstream all that uses it to spread its popularity.. How is rust when it comes to deployment. C++ is a real pain to deploy for large projects. Compilation setup is also a massive headache in c++.

steveklabnik 12y ago
With Cargo, getting a nice repeatable build is much, much easier. There's still work to be done on this front, but I'm _incredibly_ excited by this RFC: https://github.com/rust-lang/rfcs/pull/403 https://github.com/rust-lang/rfcs/pull/403

steveklabnik 12y ago

This issue on the Rust repo is very relevant: https://github.com/rust-lang/rust/issues/18085 https://github.com/rust-lang/rust/issues/18085

rando289 12y ago

I wish there was a D comparison. It seems to have similar goals.

igouy 12y ago
Make your wish reality - take the program source code and the measurement scripts and publish your own measurements. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex If you have a linux install the benchmarks game scripts should work fine, and you'd probably get help from the D community -- http://forum.dlang.org/thread/lv7h1r$mg3$2@digitalmars.com?page=3#post-juawuohxdfcxtbraejnv:40forum.dlang.org http://forum.dlang.org/thread/lv7h1r$mg3$2@digitalmars.com?p...