6 ms·

Nice to see Rust doing so well, but I'm disappointed by the performance of the Haskell code.

danieldk 13y ago
Why? I had (and often have) exactly the opposite sentiment: it is surprising that a language that is as high-level as Haskell can compete with low-level imperative languages. Regardless what one thinks of Haskell, it's a testament to the work of the GHC developers.
- mercurial 13y ago
  That's the thing about Haskell, isn't it? It offers enough guarantees that the compiler can work its magic. Which makes me think it can do better than that in terms of perfs.
  - dons 13y ago
    Every benchmark is different. And especially if you use lazy lists instead of loops with unboxed parameters. Still, it looks mostly like a test of the random number source. And also none of the nice high level optimizations are firing anyway, http://www.reddit.com/r/programming/comments/1ixnf6/benchmarking_roguelike_level_generation_go_rust/cb94j65 http://www.reddit.com/r/programming/comments/1ixnf6/benchmar...
- btmorex 13y ago
  This test isn't actually a great benchmark between languages because, as the author correctly points out, it's mostly a test of random number generation and presumably all of those languages use libc's random number generator. If you run a quick profile on the C code, you'll find the vast majority of time is spent in libc generating random numbers. To get a fair comparison without completely changing the benchmark, you should probably subtract .3-.4 seconds from each runtime as that's the amount of time spent in random number generation (again presumably all languages use libc's).
  - pcwalton 13y ago
    I would be surprised if Go used libc's random number generator; Go libraries tend to reimplement libc functionality in Go. (This is in contrast to Rust, which usually uses libc.) Indeed it is reimplemented: http://golang.org/src/pkg/math/rand/rng.go http://golang.org/src/pkg/math/rand/rng.go Rust uses a cryptographically secure random number generator (ISAAC) by default, to try to protect users from themselves. This means that the RNG is going to be slower than non-cryptographically-secure random number generators. You can get a non-cryptographic random number generator, but you have to ask for it.
    - btmorex 13y ago
      Interesting, although I think that makes the benchmark even less useful because the heart of it (generating random numbers) is not using the same algorithm across languages.
    - pbsd 13y ago
      I do believe that there's no reason that any modern language should not have a secure random number generator by default. But while ISAAC is likely fine for this purpose, I'd recommend Salsa20/12 instead (Salsa20/8 would be fine too, but let's be conservative). Salsa20 is parallelizable and can skip ahead arbitrarily, has smaller state, and is fast: on recent processors the amortized time per 32-bit integer is around 8 cycles. It was also thoroughly vetted as an eSTREAM cipher.
      - dbaupp 13y ago
        Rust has XorShift (http://en.wikipedia.org/wiki/Xorshift http://en.wikipedia.org/wiki/Xorshift) as a "secondary" RNG in the standard library, for what it's worth.
      - kibwen 13y ago
        What implementations do you suggest? I'd be happy to bring this up to the developers if there's a chance of speeding up our default RNG with one that's just as safe. Note that it must also be compatible with our MIT/Apache 2.0 dual license.
moomin 13y ago
The reports from all of these micro benchmarks seems to be consistent on the subject of Rust: it doesn't yet have the performance that justifies its existence. I'm guessing you could get similar numbers out of Java. I'm hoping Rust's numbers improve. It's incredibly promising, but without C++ level performance, it's too much semantic complexity.
- samnardoni 13y ago
  Achieving the performance that Rust does is amazing, given the focus so far been on language design and not performance. It's not even 1.0 yet! I'm sure these numbers will improve.
- pkolaczk 13y ago
  +1 The same is true probably also about D.
  - p0nce 13y ago
    No because D has C performance, now. Have you seen the blog numbers?
    - pkolaczk 13y ago
      As pointed out in the other comments on this thread, this benchmark benchmarks mostly standard RNG performance, not quality of the code generated by each compiler. Here you've got much more computationally intensive benchmark, showing that D is "as fast as C" like it could be said about Java: http://attractivechaos.wordpress.com/2013/04/06/performance-of-rust-and-dart-in-sudoku-solving/ http://attractivechaos.wordpress.com/2013/04/06/performance-... It is close, but Java is just as close, too. Not a tremendous performance advantage that someone would choose one over another for and not something I'd expect from a language marketed as "performance oriented".
      - p0nce 13y ago
        Ok you live in a world where Java is "as fast as C". Let's terminate this discussion.
logicchains 13y ago
The most recent Haskell version uses the MWC PRNG rather than Mersenne, reaching 1.05s (53% the speed of C). That's a respectable number, especially for such an expressive language. I imagine if someone could adjust it to use XorShift it would run even faster.

Why? I had (and often have) exactly the opposite sentiment: it is surprising that a language that is as high-level as Haskell can compete with low-level imperative languages. Regardless what one thinks of Haskell, it's a testament to the work of the GHC developers.

mercurial 13y ago
That's the thing about Haskell, isn't it? It offers enough guarantees that the compiler can work its magic. Which makes me think it can do better than that in terms of perfs.
- dons 13y ago
  Every benchmark is different. And especially if you use lazy lists instead of loops with unboxed parameters. Still, it looks mostly like a test of the random number source. And also none of the nice high level optimizations are firing anyway, http://www.reddit.com/r/programming/comments/1ixnf6/benchmarking_roguelike_level_generation_go_rust/cb94j65 http://www.reddit.com/r/programming/comments/1ixnf6/benchmar...
btmorex 13y ago
This test isn't actually a great benchmark between languages because, as the author correctly points out, it's mostly a test of random number generation and presumably all of those languages use libc's random number generator. If you run a quick profile on the C code, you'll find the vast majority of time is spent in libc generating random numbers. To get a fair comparison without completely changing the benchmark, you should probably subtract .3-.4 seconds from each runtime as that's the amount of time spent in random number generation (again presumably all languages use libc's).
- pcwalton 13y ago
  I would be surprised if Go used libc's random number generator; Go libraries tend to reimplement libc functionality in Go. (This is in contrast to Rust, which usually uses libc.) Indeed it is reimplemented: http://golang.org/src/pkg/math/rand/rng.go http://golang.org/src/pkg/math/rand/rng.go Rust uses a cryptographically secure random number generator (ISAAC) by default, to try to protect users from themselves. This means that the RNG is going to be slower than non-cryptographically-secure random number generators. You can get a non-cryptographic random number generator, but you have to ask for it.
  - btmorex 13y ago
    Interesting, although I think that makes the benchmark even less useful because the heart of it (generating random numbers) is not using the same algorithm across languages.
  - pbsd 13y ago
    I do believe that there's no reason that any modern language should not have a secure random number generator by default. But while ISAAC is likely fine for this purpose, I'd recommend Salsa20/12 instead (Salsa20/8 would be fine too, but let's be conservative). Salsa20 is parallelizable and can skip ahead arbitrarily, has smaller state, and is fast: on recent processors the amortized time per 32-bit integer is around 8 cycles. It was also thoroughly vetted as an eSTREAM cipher.
    - dbaupp 13y ago
      Rust has XorShift (http://en.wikipedia.org/wiki/Xorshift http://en.wikipedia.org/wiki/Xorshift) as a "secondary" RNG in the standard library, for what it's worth.
    - kibwen 13y ago
      What implementations do you suggest? I'd be happy to bring this up to the developers if there's a chance of speeding up our default RNG with one that's just as safe. Note that it must also be compatible with our MIT/Apache 2.0 dual license.

moomin 13y ago

The reports from all of these micro benchmarks seems to be consistent on the subject of Rust: it doesn't yet have the performance that justifies its existence. I'm guessing you could get similar numbers out of Java. I'm hoping Rust's numbers improve. It's incredibly promising, but without C++ level performance, it's too much semantic complexity.

samnardoni 13y ago
Achieving the performance that Rust does is amazing, given the focus so far been on language design and not performance. It's not even 1.0 yet! I'm sure these numbers will improve.
pkolaczk 13y ago
+1 The same is true probably also about D.
- p0nce 13y ago
  No because D has C performance, now. Have you seen the blog numbers?
  - pkolaczk 13y ago
    As pointed out in the other comments on this thread, this benchmark benchmarks mostly standard RNG performance, not quality of the code generated by each compiler. Here you've got much more computationally intensive benchmark, showing that D is "as fast as C" like it could be said about Java: http://attractivechaos.wordpress.com/2013/04/06/performance-of-rust-and-dart-in-sudoku-solving/ http://attractivechaos.wordpress.com/2013/04/06/performance-... It is close, but Java is just as close, too. Not a tremendous performance advantage that someone would choose one over another for and not something I'd expect from a language marketed as "performance oriented".
    - p0nce 13y ago
      Ok you live in a world where Java is "as fast as C". Let's terminate this discussion.

logicchains 13y ago

The most recent Haskell version uses the MWC PRNG rather than Mersenne, reaching 1.05s (53% the speed of C). That's a respectable number, especially for such an expressive language. I imagine if someone could adjust it to use XorShift it would run even faster.