6 ms·

Just to note, these benchmarks are biased. This is a competition between highly-tuned-to-the-processor Julia with tuned BLAS vs other languages out-of-the-box b

Just to note, these benchmarks are biased. This is a competition between highly-tuned-to-the-processor Julia with tuned BLAS vs other languages out-of-the-box binaries. Additionally, the benchmarks tests are written inefficiently in the comparison languages. But even then Javascript comes out on top on some benchmark tests.

dnautics 9y ago
That's because the jit compiler in the Julia standard library selects the blas for you. As a programmer you don't have to optimize beyond using idiomatic type strategies. You could, say write a galois field type and use the \ solve operator to solve a Reed Solomon Erasure using old fashioned lu decomposition (automatically without pivoting, even), and then turn around and use the same \ operator with floats one line down and it will give you highly optimized blas.
ViralBShah 9y ago
Only one of the benchmarks is BLAS dependent and the point is to make sure that it is easy to get a tuned BLAS in that system. If these benchmarks have resulted in all the languages focusing on a fast BLAS, I think that is fantastic.
3JPLW 9y ago
Yes, these benchmarks do tend to be a lightning-rod for controversy. You have two complaints: 1. Yes, some languages do not make it easy to install and link against a well-tuned BLAS. The benchmarks were very recently updated, and you can see the level of effort required to get everything installed and working properly. This was all done in the open on the discourse message board. 2. Yes, the benchmarks involve testing very specific language-level features like iteration, recursion, and IO. And they're testing the ability of specific languages to do these things. Yes, of course in Python you'd look towards a vectorized library call instead of a `for` loop, but then you're not testing Python itself anymore.
- bjourne 9y ago
  How come there are no entries for Julia on the Benchmarks Game? http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/ Given Julia's good performance, it is a little strange. :)
  - ihnorton 9y ago
    Inclusion is by selection of the admin [1,2]. There are Julia implementations available for most of the benchmarks [3] so perhaps eventually they will be included. [1] https://alioth.debian.org/forum/message.php?msg_id=182495&group_id=100815 https://alioth.debian.org/forum/message.php?msg_id=182495&gr... [2] http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex [3] https://github.com/JuliaLang/julia/tree/master/test/perf/shootout https://github.com/JuliaLang/julia/tree/master/test/perf/sho...
    - bjourne 9y ago
      Interesting! Does that mean the Benchmarks Game is not open for new languages? The faq entry is kind of ambiguous.
      - igouy 9y ago
        It means -- "If you're interested in something not shown on the benchmarks game website then please take the program source code and the measurement scripts and publish your own measurements." Like this guy did -- https://pybenchmarks.org/ https://pybenchmarks.org/

dnautics 9y ago

That's because the jit compiler in the Julia standard library selects the blas for you. As a programmer you don't have to optimize beyond using idiomatic type strategies. You could, say write a galois field type and use the \ solve operator to solve a Reed Solomon Erasure using old fashioned lu decomposition (automatically without pivoting, even), and then turn around and use the same \ operator with floats one line down and it will give you highly optimized blas.

ViralBShah 9y ago

Only one of the benchmarks is BLAS dependent and the point is to make sure that it is easy to get a tuned BLAS in that system. If these benchmarks have resulted in all the languages focusing on a fast BLAS, I think that is fantastic.

3JPLW 9y ago

Yes, these benchmarks do tend to be a lightning-rod for controversy. You have two complaints: 1. Yes, some languages do not make it easy to install and link against a well-tuned BLAS. The benchmarks were very recently updated, and you can see the level of effort required to get everything installed and working properly. This was all done in the open on the discourse message board. 2. Yes, the benchmarks involve testing very specific language-level features like iteration, recursion, and IO. And they're testing the ability of specific languages to do these things. Yes, of course in Python you'd look towards a vectorized library call instead of a `for` loop, but then you're not testing Python itself anymore.

bjourne 9y ago
How come there are no entries for Julia on the Benchmarks Game? http://benchmarksgame.alioth.debian.org/ http://benchmarksgame.alioth.debian.org/ Given Julia's good performance, it is a little strange. :)
- ihnorton 9y ago
  Inclusion is by selection of the admin [1,2]. There are Julia implementations available for most of the benchmarks [3] so perhaps eventually they will be included. [1] https://alioth.debian.org/forum/message.php?msg_id=182495&group_id=100815 https://alioth.debian.org/forum/message.php?msg_id=182495&gr... [2] http://benchmarksgame.alioth.debian.org/play.html#languagex http://benchmarksgame.alioth.debian.org/play.html#languagex [3] https://github.com/JuliaLang/julia/tree/master/test/perf/shootout https://github.com/JuliaLang/julia/tree/master/test/perf/sho...
  - bjourne 9y ago
    Interesting! Does that mean the Benchmarks Game is not open for new languages? The faq entry is kind of ambiguous.
    - igouy 9y ago
      It means -- "If you're interested in something not shown on the benchmarks game website then please take the program source code and the measurement scripts and publish your own measurements." Like this guy did -- https://pybenchmarks.org/ https://pybenchmarks.org/