6 ms·

It seems to me that there are a lot of apples-to-oranges comparisons here? Some implementations are using the language's standard library hashtable implementati

It seems to me that there are a lot of apples-to-oranges comparisons here? Some implementations are using the language's standard library hashtable implementation while others are using 3rd party version (with different algorithms and data structures across all of them), some are using multiple threads while others are single threaded etc. As a result, I wouldn't read too much into the rankings you see here.

IshKebab 10y ago
That's why they call it a game.
- igouy 10y ago
  http://benchmarksgame.alioth.debian.org/sometimes-people-just-make-up-stuff.html#name-game http://benchmarksgame.alioth.debian.org/sometimes-people-jus...
  - IshKebab 10y ago
    Huh, I could have sworn at the time that that was the reason but C2 backs it up: http://wiki.c2.com/?GreatComputerLanguageShootout http://wiki.c2.com/?GreatComputerLanguageShootout Maybe they should remove 'game' from the title if they need an FAQ for it. Or even better they should acknowledge that it is a game.
    - igouy 10y ago
      The word game has several different meanings -- not all of them are dismissive.
spoiler 10y ago
> It seems to me that there are a lot of apples-to-oranges comparisons here? So, the usual. :) I stopped taking into account the benchmarks game completely after I saw apples-to-potatoes comparisons a few years back.
- igouy 10y ago
  http://benchmarksgame.alioth.debian.org/dont-jump-to-conclusions.html#apples-and-oranges http://benchmarksgame.alioth.debian.org/dont-jump-to-conclus... Long ago the Ada program contributors told me -- It's only the same program if the same assembler is generated ;-)
imaginenore 10y ago
So? You're welcome to write your version, if you think you can make a faster one.
adrianN 10y ago
There was some ruckus regarding the use of standard-library hash tables. See this thread https://news.ycombinator.com/item?id=13266801 https://news.ycombinator.com/item?id=13266801 for some links.
bluejekyll 10y ago
And some languages are allowed to use FFI to make their impl faster. There's some rule about this that I don't understand, but oh well. It's all for fun, not serious. But come on, now Rust can legitimately be called "faster than C" ;) At least until the Clang C version is added... or maybe it will still be faster.
- coldtea 10y ago
  >And some languages are allowed to use FFI to make their impl faster. There's some rule about this that I don't understand, but oh well. It's all for fun, not serious. Actually it's pretty easy: if you can write a faster version in any language, given whatever is there in the language, even if it's c implemented standard library stuff, do it. The implementations are not meant to be final -- people can contribute faster ones.
- emn13 10y ago
  I suspect all languages without exception use some standard library functionality in at least a few of those sample programs, and most "standard" libraries aren't constrained to be self-hosting - so all of em use some native code, probably written in C or C++. I suspect that's true of rust too. FFI is a fact of life. I can imagine it would be perverting the intent of the game if you explicitly used FFI to delegate the actual core of the benchmark program to C as opposed to using "standard" building blocks, but the distinciton is necessarily vague.
  - steveklabnik 10y ago
    > I suspect that's true of rust too. rustc uses LLVM and jemalloc (by default). Other than that, it is all Rust.
igouy 10y ago
Correct! http://benchmarksgame.alioth.debian.org/u64q/knucleotide-description.html#knucleotide http://benchmarksgame.alioth.debian.org/u64q/knucleotide-des...
88e282102ae2e5b 10y ago
Sure, but I'm less worried about absolute rank and more interested in whether the language I'm using is on the order of C/C++/Fortran. I can sell a 2x slowdown to my boss if I can show that that's the only cost of a significantly more elegant and productive language, but at 100x that's a much harder sell.
- rbehrends 10y ago
  Just be careful to remember that good performance on microbenchmarks does not necessarily translate to good performance on large applications. Some factors to consider are: 1. Cache performance and locality may differ considerably for large applications. For example, a microbenchmark may perform well because it can monopolize the L1 cache in a way that is not possible for larger applications. 2. Aggressive inlining, a common factor in microbenchmark performance, may not scale well for large code bases. The tradeoffs here can be rather complex: https://webkit.org/blog/2826/unusual-speed-boost-size-matters/ https://webkit.org/blog/2826/unusual-speed-boost-size-matter... 3. Microbenchmarks often avoid abstraction in order to achieve high performance. This can create unacceptable software engineering costs.
  - achamayou 10y ago
    It's still useful as a potential filter. If someone's implemented your microbenchmark in langB and it's within 2x of langA, then maybe they're worth comparing. If on the other hand langB is 20x away from langA in microbenchmarks, you can be pretty sure that it won't ever be a good replacement.
    - nostrademons 10y ago
      As long as you have people of roughly equivalent skill level within their language implementing the microbenchmarks. It's not that hard to get a 10x or even 100x speed improvement in the same language when an expert rewrites the naive code that a newbie wrote.
      - 10y ago
igouy 10y ago
> some are using multiple threads while others are single threaded Perhaps sort by cpu (seconds) rather than (elapsed) secs. http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=knucleotide&sort=fullcpu http://benchmarksgame.alioth.debian.org/u64q/performance.php...

IshKebab 10y ago

That's why they call it a game.

igouy 10y ago
http://benchmarksgame.alioth.debian.org/sometimes-people-just-make-up-stuff.html#name-game http://benchmarksgame.alioth.debian.org/sometimes-people-jus...
- IshKebab 10y ago
  Huh, I could have sworn at the time that that was the reason but C2 backs it up: http://wiki.c2.com/?GreatComputerLanguageShootout http://wiki.c2.com/?GreatComputerLanguageShootout Maybe they should remove 'game' from the title if they need an FAQ for it. Or even better they should acknowledge that it is a game.
  - igouy 10y ago
    The word game has several different meanings -- not all of them are dismissive.

spoiler 10y ago

> It seems to me that there are a lot of apples-to-oranges comparisons here? So, the usual. :) I stopped taking into account the benchmarks game completely after I saw apples-to-potatoes comparisons a few years back.

igouy 10y ago
http://benchmarksgame.alioth.debian.org/dont-jump-to-conclusions.html#apples-and-oranges http://benchmarksgame.alioth.debian.org/dont-jump-to-conclus... Long ago the Ada program contributors told me -- It's only the same program if the same assembler is generated ;-)

imaginenore 10y ago

So? You're welcome to write your version, if you think you can make a faster one.

adrianN 10y ago

There was some ruckus regarding the use of standard-library hash tables. See this thread https://news.ycombinator.com/item?id=13266801 https://news.ycombinator.com/item?id=13266801 for some links.

bluejekyll 10y ago

And some languages are allowed to use FFI to make their impl faster. There's some rule about this that I don't understand, but oh well. It's all for fun, not serious. But come on, now Rust can legitimately be called "faster than C" ;) At least until the Clang C version is added... or maybe it will still be faster.

coldtea 10y ago
>And some languages are allowed to use FFI to make their impl faster. There's some rule about this that I don't understand, but oh well. It's all for fun, not serious. Actually it's pretty easy: if you can write a faster version in any language, given whatever is there in the language, even if it's c implemented standard library stuff, do it. The implementations are not meant to be final -- people can contribute faster ones.
emn13 10y ago
I suspect all languages without exception use some standard library functionality in at least a few of those sample programs, and most "standard" libraries aren't constrained to be self-hosting - so all of em use some native code, probably written in C or C++. I suspect that's true of rust too. FFI is a fact of life. I can imagine it would be perverting the intent of the game if you explicitly used FFI to delegate the actual core of the benchmark program to C as opposed to using "standard" building blocks, but the distinciton is necessarily vague.
- steveklabnik 10y ago
  > I suspect that's true of rust too. rustc uses LLVM and jemalloc (by default). Other than that, it is all Rust.

igouy 10y ago

Correct! http://benchmarksgame.alioth.debian.org/u64q/knucleotide-description.html#knucleotide http://benchmarksgame.alioth.debian.org/u64q/knucleotide-des...

88e282102ae2e5b 10y ago

Sure, but I'm less worried about absolute rank and more interested in whether the language I'm using is on the order of C/C++/Fortran. I can sell a 2x slowdown to my boss if I can show that that's the only cost of a significantly more elegant and productive language, but at 100x that's a much harder sell.

rbehrends 10y ago
Just be careful to remember that good performance on microbenchmarks does not necessarily translate to good performance on large applications. Some factors to consider are: 1. Cache performance and locality may differ considerably for large applications. For example, a microbenchmark may perform well because it can monopolize the L1 cache in a way that is not possible for larger applications. 2. Aggressive inlining, a common factor in microbenchmark performance, may not scale well for large code bases. The tradeoffs here can be rather complex: https://webkit.org/blog/2826/unusual-speed-boost-size-matters/ https://webkit.org/blog/2826/unusual-speed-boost-size-matter... 3. Microbenchmarks often avoid abstraction in order to achieve high performance. This can create unacceptable software engineering costs.
- achamayou 10y ago
  It's still useful as a potential filter. If someone's implemented your microbenchmark in langB and it's within 2x of langA, then maybe they're worth comparing. If on the other hand langB is 20x away from langA in microbenchmarks, you can be pretty sure that it won't ever be a good replacement.
  - nostrademons 10y ago
    As long as you have people of roughly equivalent skill level within their language implementing the microbenchmarks. It's not that hard to get a 10x or even 100x speed improvement in the same language when an expert rewrites the naive code that a newbie wrote.
    - 10y ago

igouy 10y ago

> some are using multiple threads while others are single threaded Perhaps sort by cpu (seconds) rather than (elapsed) secs. http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=knucleotide&sort=fullcpu http://benchmarksgame.alioth.debian.org/u64q/performance.php...