5 ms·

See "The Computer Language Benchmarks Game" at http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/ for a head-to-head comparison o

by dalke 11y ago

See "The Computer Language Benchmarks Game" at http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/ for a head-to-head comparison of various small programs as measured across a wide number of language implementations. Do be careful that a "language" has no performance, only implementations of the language. For example, different implementations of Python may differ by an order of magnitude on a given task, as might different C compilers. Java and Javascript used to be considered slow languages, but the effective billions of dollars of R&D improved the implementations. The overall solution space is complex. For example, development time on Python is about 1/2 that of C++ (the classic paper is http://page.mi.fu-berlin.de/prechelt/Biblio/jccpprt_computer2000.pdf http://page.mi.fu-berlin.de/prechelt/Biblio/jccpprt_computer... - is there something more recent?). Since algorithms can trump raw performance, it may be that in 8 hours its possible to develop a more sophisticated implementation in Python which outperforms the naive implementation in C++. In the late 1990s, when people started using Python for traditional supercomputing problems, the phrase was to use Python to "steer" the low-level code written in a faster language. For example, NumPy is a Python API which uses C/ Fortran/ assembly code at lower levels for doing array calculations. Thus, only some people need to be skilled in both levels, while most can focus on writing Python code. The actual code has parts written in multiple languages, and not "only C/C++".

orf 11y ago
Didn't the PyPy folks have some issue with the language shootout, which led them to make their own?[1][2]. IIRC a lot of the Python code wasn't that optimized at all compared to the C/C++ programs. 1. http://speed.pypy.org/ http://speed.pypy.org/ 2. https://alexgaynor.net/2011/apr/03/my-experience-computer-language-shootout/ https://alexgaynor.net/2011/apr/03/my-experience-computer-la...
- deleted 11y ago
  [deleted]
- igouy 11y ago
  No. Programs optimized specifically for PyPy were already being shown on the benchmarks game. Previously, factual corrections to that blog post were shown in the comments but now the comments are not shown.
- igouy 11y ago
  [2] The blog post tell's us that Alex Gaynor confirmed "with some CPython core developers" that his program didn't work because of a bug in CPython. But the blog post doesn't tell us that Alex Gaynor never said there was any problem with a CPython bug. Alex Gaynor's blog post tell's us that "It's also not possible to send any messages once your ticket has been marked as closed, meaning to dispute a decision you basically need to pray the maintainer reopens it for some reason." But that's completely untrue! You can send messages when the ticket is marked closed! And you can open topics in the public forum! And you can click on a username and send email in 2 clicks. There just wouldn't be any story to blog about, if Alex Gaynor admitted that he could easily have told me -- the bug is in CPython not in my program, so show my program -- but chose to say nothing.
vardump 11y ago
I think "The Computer Language Benchmarks Game" is so politically loaded, that it shouldn't be referred anymore. It can give people very wrong idea of performance between different languages. Especially people who don't have significant software development experience.
- CyberDildonics 11y ago
  What is a better resource? The page itself talks about how no one should put much stock in the results, but there aren't really alternatives that give the same type of information.
- chestervonwinch 11y ago
  > so politically loaded why do you say this?
  - vardump 11y ago
    That benchmark does not allow faster implementations of languages to be part of the benchmark. Like they allow only interpreted implementation of certain languages, not fast JIT version. It's just the end results are even less meaningful this way. All this is entirely within their rights, of course. Someone else should create a competing programming language benchmark that includes all reasonable language implementations and all reasonable benchmark entries in those languages. Unfortunately, no one has bothered.
    - chestervonwinch 11y ago
      So you believe the reason for this is because of some political agenda? Are there conflicts of interest with the maintainers of the site? or I guess what are your reasons for your believing this? I'm genuinely asking / curious. I've only looked at the site in passing maybe a few times.
    - igouy 11y ago
      Please take the program source code and measurement scripts and publish your own measurements. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex
igouy 11y ago
Also see https://www.plat-forms.org/ https://www.plat-forms.org/

by dalke 11y ago

orf 11y ago

Didn't the PyPy folks have some issue with the language shootout, which led them to make their own?[1][2]. IIRC a lot of the Python code wasn't that optimized at all compared to the C/C++ programs. 1. http://speed.pypy.org/ http://speed.pypy.org/ 2. https://alexgaynor.net/2011/apr/03/my-experience-computer-language-shootout/ https://alexgaynor.net/2011/apr/03/my-experience-computer-la...

deleted 11y ago
[deleted]
igouy 11y ago
No. Programs optimized specifically for PyPy were already being shown on the benchmarks game. Previously, factual corrections to that blog post were shown in the comments but now the comments are not shown.
igouy 11y ago
[2] The blog post tell's us that Alex Gaynor confirmed "with some CPython core developers" that his program didn't work because of a bug in CPython. But the blog post doesn't tell us that Alex Gaynor never said there was any problem with a CPython bug. Alex Gaynor's blog post tell's us that "It's also not possible to send any messages once your ticket has been marked as closed, meaning to dispute a decision you basically need to pray the maintainer reopens it for some reason." But that's completely untrue! You can send messages when the ticket is marked closed! And you can open topics in the public forum! And you can click on a username and send email in 2 clicks. There just wouldn't be any story to blog about, if Alex Gaynor admitted that he could easily have told me -- the bug is in CPython not in my program, so show my program -- but chose to say nothing.

vardump 11y ago

I think "The Computer Language Benchmarks Game" is so politically loaded, that it shouldn't be referred anymore. It can give people very wrong idea of performance between different languages. Especially people who don't have significant software development experience.

CyberDildonics 11y ago
What is a better resource? The page itself talks about how no one should put much stock in the results, but there aren't really alternatives that give the same type of information.
chestervonwinch 11y ago
> so politically loaded why do you say this?
- vardump 11y ago
  That benchmark does not allow faster implementations of languages to be part of the benchmark. Like they allow only interpreted implementation of certain languages, not fast JIT version. It's just the end results are even less meaningful this way. All this is entirely within their rights, of course. Someone else should create a competing programming language benchmark that includes all reasonable language implementations and all reasonable benchmark entries in those languages. Unfortunately, no one has bothered.
  - chestervonwinch 11y ago
    So you believe the reason for this is because of some political agenda? Are there conflicts of interest with the maintainers of the site? or I guess what are your reasons for your believing this? I'm genuinely asking / curious. I've only looked at the site in passing maybe a few times.
  - igouy 11y ago
    Please take the program source code and measurement scripts and publish your own measurements. http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex

igouy 11y ago

Also see https://www.plat-forms.org/ https://www.plat-forms.org/