5 ms·

This is a rather meaningless comparison, since the differences are going to be dominated by: 1) The choice of libraries/datatypes used for strings and word->co

This is a rather meaningless comparison, since the differences are going to be dominated by: 1) The choice of libraries/datatypes used for strings and word->count map 2) How the source file is split into words - probably library function again, although in C/C++ one could choose to implement a super-optimized low level version that would blow the others away IMO a performance comparison between languages is only meaningful if the runtime is not dominated by library functions, or if one admits it's really a standard library performance comparison, not a language comparison.

ashvardanian 4y ago
Agreed. It’s a common desire to boil benchmarks down to a single number. Some of those languages, however, are far more complex then the others. And as any complex tool, it requires certain methodology. If you are to use heavy machinery like C++, forget iostreams and character-level processing on your hot paths.
kitd 4y ago
I understand what you're saying. However if you look at it not from the point of the performance figures per se, but rather which language, when written idiomatically, can solve a realistic problem performantly, then I think it makes more sense.
- HarHarVeryFunny 4y ago
  True - there's still value in these types of comparison, although the click-baity "C++ vs Python" title rather detracts from that. I'm not sure how much value there is in this specific benchmark though since in the real world you'd be using a utility to do this (e.g. linux wc - word count) and it's not obvious what's dominating the runtime (I'd guess reading file and/or splitting into words), so what you're takeaway should be (idiomatic X is good for what, exactly?) if wanting to extrapolate this to some other task. For that matter, in the real world people don't really choose language on a task specific basis... It's either use language X because the project demands it, or write some limited use utility in a scripting language if performance isn't a major concern.
oconnor663 4y ago
Do you have a particular example problem in mind that might demonstrate language performance better? One of the Benchmark Game programs maybe?
- camel-cdr 4y ago
  What is there to benchmark between C++, C, Rust, or any other low level compiled language though? With any one of them you can essentially achieve the same assembly code generation, if you try hard enough. In the end it either boils down to compiler optimisations or library implementations.
  - jamincan 4y ago
    I think the interesting question there (and really any language comparison) is to compare the result of idiomatic implementations. Unfortunately, there is no clear answer on what is idiomatic, even with something like Python that is more opinionated than most on the question.
    - igouy 4y ago
      When there is "no clear answer on what is idiomatic", we are left with uninteresting "yes it is" / "no it isn't" Dead Parrot assertions. http://montypython.50webs.com/scripts/Series_1/53.htm http://montypython.50webs.com/scripts/Series_1/53.htm
      - oconnor663 4y ago
        I think the best way forward on the idiom question is to accept that we can't have a competitive/adversarial benchmark suite where idiom is a factor. I think you really need one author, or maybe a group of colleagues who trust each other, to write the whole suite and make their own choices about what idiom means to them. Folks who disagree with those choices can either write an article about how much performance you gain from making what changes, or just produce their own whole suite. Having a very high level comparison like "this is how languages stack up with these idiomatic choices, but this is a different chart with different choices" would be interesting, even though it would take more work to interpret it.
Scarbutt 4y ago
exactly, is mostly comparing different C libraries implementations.
kovac 4y ago
With C I rarely use libs outside of stdlib and when I do I pay attention to performance more than convenience. I think even if it's dominated by libs and data types I think it has value as it might reflect what actually happens out there in the wild rather than a purely academic exercise.
gcanyon 4y ago
My language of choice, LiveCode, is a superset of HyperTalk. The concept of "words" has been built-in (not a library) since the '80s.

ashvardanian 4y ago

Agreed. It’s a common desire to boil benchmarks down to a single number. Some of those languages, however, are far more complex then the others. And as any complex tool, it requires certain methodology. If you are to use heavy machinery like C++, forget iostreams and character-level processing on your hot paths.

kitd 4y ago

I understand what you're saying. However if you look at it not from the point of the performance figures per se, but rather which language, when written idiomatically, can solve a realistic problem performantly, then I think it makes more sense.

HarHarVeryFunny 4y ago
True - there's still value in these types of comparison, although the click-baity "C++ vs Python" title rather detracts from that. I'm not sure how much value there is in this specific benchmark though since in the real world you'd be using a utility to do this (e.g. linux wc - word count) and it's not obvious what's dominating the runtime (I'd guess reading file and/or splitting into words), so what you're takeaway should be (idiomatic X is good for what, exactly?) if wanting to extrapolate this to some other task. For that matter, in the real world people don't really choose language on a task specific basis... It's either use language X because the project demands it, or write some limited use utility in a scripting language if performance isn't a major concern.

oconnor663 4y ago

Do you have a particular example problem in mind that might demonstrate language performance better? One of the Benchmark Game programs maybe?

camel-cdr 4y ago
What is there to benchmark between C++, C, Rust, or any other low level compiled language though? With any one of them you can essentially achieve the same assembly code generation, if you try hard enough. In the end it either boils down to compiler optimisations or library implementations.
- jamincan 4y ago
  I think the interesting question there (and really any language comparison) is to compare the result of idiomatic implementations. Unfortunately, there is no clear answer on what is idiomatic, even with something like Python that is more opinionated than most on the question.
  - igouy 4y ago
    When there is "no clear answer on what is idiomatic", we are left with uninteresting "yes it is" / "no it isn't" Dead Parrot assertions. http://montypython.50webs.com/scripts/Series_1/53.htm http://montypython.50webs.com/scripts/Series_1/53.htm
    - oconnor663 4y ago
      I think the best way forward on the idiom question is to accept that we can't have a competitive/adversarial benchmark suite where idiom is a factor. I think you really need one author, or maybe a group of colleagues who trust each other, to write the whole suite and make their own choices about what idiom means to them. Folks who disagree with those choices can either write an article about how much performance you gain from making what changes, or just produce their own whole suite. Having a very high level comparison like "this is how languages stack up with these idiomatic choices, but this is a different chart with different choices" would be interesting, even though it would take more work to interpret it.

Scarbutt 4y ago

exactly, is mostly comparing different C libraries implementations.

kovac 4y ago

With C I rarely use libs outside of stdlib and when I do I pay attention to performance more than convenience. I think even if it's dominated by libs and data types I think it has value as it might reflect what actually happens out there in the wild rather than a purely academic exercise.

gcanyon 4y ago

My language of choice, LiveCode, is a superset of HyperTalk. The concept of "words" has been built-in (not a library) since the '80s.