3 ms·

> Generally, the point of language benchmarks is to show how the languages compare at solving the same problem, without external libraries. If this were the on

> Generally, the point of language benchmarks is to show how the languages compare at solving the same problem, without external libraries. If this were the only concern, it should be valid to create a blob of binary and call that function for the optimal performance. (Python's ctypes makes this very easy, for example.) So you want an idiomatic solution instead, and Numpy for matrix computation is considered idiomatic in Python, even more than the pure Python code.

tsimionescu 3y ago
No, creating a binary blob is not a realistic option, and it's still not a Python/Java/C++ whatever solution. It would be handwritten assembly that you use the FFI facilities to call into. So, it's not fair game in any honest benchmark. And even if using a C library is idiomatic Python, it still has no place in a language benchmark. It's a C library, not a Python implementation.
- lifthrasiir 3y ago
  > No, creating a binary blob is not a realistic option, and it's still not a Python/Java/C++ whatever solution. Once again, "realistic" is subjective and I would say no "realistic" user will try to multiply arbitrarily-sized matrices in pure Python. (I can see small enough matrices, like 3x3 or 4x4, might be different.) And... > So, it's not fair game in any honest benchmark. And even if using a C library is idiomatic Python, it still has no place in a language benchmark. It's a C library, not a Python implementation. ...you have correctly figured out that it's unfair for anyone using your (vague) definition of "realistic". It's also true that your proposal is also unfair for anyone using my definition of "idiomatic" however. I can try to defend my definition with quantative arguments, but I don't feel like doing so. In fact I tend to ignore most "language benchmarks" because it is virtually impossible to make them reasonably fair. This one is no exception. Best "language benchmarks" tend to be more like language showcases with useful commentaries, there will be no single winner but you will get a good sense of pros and cons of each language-implementation-strategy combination. They are generally not advertised as "benchmarks", of course.
  - deleted 3y ago
    [deleted]
- igouy 3y ago
  How else would you find the difference between using that library from C and using that library from Python? pidigits gcc #1 https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/pidigits-gcc-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... pidigits Python3 #3 https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/pidigits-python3-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

tsimionescu 3y ago

No, creating a binary blob is not a realistic option, and it's still not a Python/Java/C++ whatever solution. It would be handwritten assembly that you use the FFI facilities to call into. So, it's not fair game in any honest benchmark. And even if using a C library is idiomatic Python, it still has no place in a language benchmark. It's a C library, not a Python implementation.

lifthrasiir 3y ago
> No, creating a binary blob is not a realistic option, and it's still not a Python/Java/C++ whatever solution. Once again, "realistic" is subjective and I would say no "realistic" user will try to multiply arbitrarily-sized matrices in pure Python. (I can see small enough matrices, like 3x3 or 4x4, might be different.) And... > So, it's not fair game in any honest benchmark. And even if using a C library is idiomatic Python, it still has no place in a language benchmark. It's a C library, not a Python implementation. ...you have correctly figured out that it's unfair for anyone using your (vague) definition of "realistic". It's also true that your proposal is also unfair for anyone using my definition of "idiomatic" however. I can try to defend my definition with quantative arguments, but I don't feel like doing so. In fact I tend to ignore most "language benchmarks" because it is virtually impossible to make them reasonably fair. This one is no exception. Best "language benchmarks" tend to be more like language showcases with useful commentaries, there will be no single winner but you will get a good sense of pros and cons of each language-implementation-strategy combination. They are generally not advertised as "benchmarks", of course.
- deleted 3y ago
  [deleted]
igouy 3y ago
How else would you find the difference between using that library from C and using that library from Python? pidigits gcc #1 https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/pidigits-gcc-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... pidigits Python3 #3 https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/pidigits-python3-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...