5 ms·

Most the Debian benchmarks for C# are cheaty too. They frequently just call out to C libraries rather than use something implemented in the language.

by vips7L 2y ago

neonsunset 2y ago
Not sure which benchmarks you have in mind. Could you provide a link to any of those? .NET's standard library never calls into anything C aside from kernel APIs and certain runtime helpers which is a given. If you meant BenchmarksGame, then it's the other way around - Java is most competitive where it relies heavily on GC[0], and loses in other areas which require capability to write a low-level implementation[1] that C# provides. The only places where there are C calls are pidigts[2] and regex-redux[3] benchmarks, in both of which Java submissions have to import pre-generated or pre-made bindings to GMP and PCRE2 respectively. As do all other languages, with varying degrees of "preparation". [0]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-java-7.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [1]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/spectralnorm-csharpcore-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [2]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/pidigits-java-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [3]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-java-8.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- vips7L 2y ago
  Im sorry, but calling out to C libraries — regardless of the language — is cheating. Just because everyone in the competition is on steroids doesn’t mean you got there legitimately.
  - neonsunset 2y ago
    This is a strange reply given that sibling comment points out it's only 2 out of 10 benchmarks where this is allowed because all languages end up calling out to the same libraries. Even if you prohibit PCRE2, the .NET submissions using out-of-box Regex engine end up being about 4 times faster than Java. Surprisingly, even though .NET's BigInteger is known for its inefficiency, it ends up being more memory efficient and marginally faster at pidigits than a Java submission that does not use GMP. The implementations are not line-by-line equivalent so may not be perfectly representative of performance of each BigInt implementation. My point being - if you look at the submissions closer, the data gives much clearer picture and only supports the argument that C# is a very usable language for solving the tasks one would usually reach for C, C++ or Rust instead.
    - vips7L 2y ago
      Its not a strange reply at all. _All_ of those languages are cheating. Those benchmarks are junk because they don't test implementations in the language.
      - igouy 2y ago
        https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-java-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Sure looks like it's written in Java!
  - igouy 2y ago
    https://www.wilfred.me.uk/blog/2014/10/20/the-fastest-bigint-in-the-west/ https://www.wilfred.me.uk/blog/2014/10/20/the-fastest-bigint... Look at all the programming language implementations that provide big integers by calling out to GMP. Why would it be "cheating" when available to all and done openly? Libraries matter.
igouy 2y ago
No, they do not "frequently just call out to C libraries". 2 of 10 (pidigits and regex-redux) allow use of widely available third party libraries — GMP, PCRE, RE2 — because there were language implementations that simply wrapped those libraries.
- lossolo 2y ago
  You should really create a way to filter solutions with SIMD intrinsics and without them.
  - igouy 2y ago
    Like the hand-written vector instructions | "unsafe" section down-page? https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... It’s not that easy. I assume other programs hide the use in macros and libraries, in ways far beyond my simple understanding.
    - lossolo 2y ago
      Cool! I didn’t see it before, maybe because I mostly use the language vs. language feature. There is no such section there, but it would be very helpful IMO, instead of clicking every solution to check which one is intrinsics free.
      - igouy 2y ago
        > language vs. language Where there are few enough programs that readers should check that the programs they compare seem appropriate for their purpose.
        lossolo 2y ago
        Well, it's your site, so you can do what you want with it, but I don't believe what you just wrote is logical at all. Sometimes, you just want to see, in general, how one language compares to another when one uses intrinsics and the other doesn't, without having to click through every single benchmark across multiple versions to find one without intrinsics. This is just bad UX and waste of time.

Most the Debian benchmarks for C# are cheaty too. They frequently just call out to C libraries rather than use something implemented in the language.

by vips7L 2y ago

Most the Debian benchmarks for C# are cheaty too. They frequently just call out to C libraries rather than use something implemented in the language.

neonsunset 2y ago

Not sure which benchmarks you have in mind. Could you provide a link to any of those? .NET's standard library never calls into anything C aside from kernel APIs and certain runtime helpers which is a given. If you meant BenchmarksGame, then it's the other way around - Java is most competitive where it relies heavily on GC[0], and loses in other areas which require capability to write a low-level implementation[1] that C# provides. The only places where there are C calls are pidigts[2] and regex-redux[3] benchmarks, in both of which Java submissions have to import pre-generated or pre-made bindings to GMP and PCRE2 respectively. As do all other languages, with varying degrees of "preparation". [0]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-java-7.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [1]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/spectralnorm-csharpcore-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [2]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/pidigits-java-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [3]: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/regexredux-java-8.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

vips7L 2y ago
Im sorry, but calling out to C libraries — regardless of the language — is cheating. Just because everyone in the competition is on steroids doesn’t mean you got there legitimately.
- neonsunset 2y ago
  This is a strange reply given that sibling comment points out it's only 2 out of 10 benchmarks where this is allowed because all languages end up calling out to the same libraries. Even if you prohibit PCRE2, the .NET submissions using out-of-box Regex engine end up being about 4 times faster than Java. Surprisingly, even though .NET's BigInteger is known for its inefficiency, it ends up being more memory efficient and marginally faster at pidigits than a Java submission that does not use GMP. The implementations are not line-by-line equivalent so may not be perfectly representative of performance of each BigInt implementation. My point being - if you look at the submissions closer, the data gives much clearer picture and only supports the argument that C# is a very usable language for solving the tasks one would usually reach for C, C++ or Rust instead.
  - vips7L 2y ago
    Its not a strange reply at all. _All_ of those languages are cheating. Those benchmarks are junk because they don't test implementations in the language.
    - igouy 2y ago
      https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/nbody-java-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Sure looks like it's written in Java!
- igouy 2y ago
  https://www.wilfred.me.uk/blog/2014/10/20/the-fastest-bigint-in-the-west/ https://www.wilfred.me.uk/blog/2014/10/20/the-fastest-bigint... Look at all the programming language implementations that provide big integers by calling out to GMP. Why would it be "cheating" when available to all and done openly? Libraries matter.

igouy 2y ago

No, they do not "frequently just call out to C libraries". 2 of 10 (pidigits and regex-redux) allow use of widely available third party libraries — GMP, PCRE, RE2 — because there were language implementations that simply wrapped those libraries.

lossolo 2y ago
You should really create a way to filter solutions with SIMD intrinsics and without them.
- igouy 2y ago
  Like the hand-written vector instructions | "unsafe" section down-page? https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... It’s not that easy. I assume other programs hide the use in macros and libraries, in ways far beyond my simple understanding.
  - lossolo 2y ago
    Cool! I didn’t see it before, maybe because I mostly use the language vs. language feature. There is no such section there, but it would be very helpful IMO, instead of clicking every solution to check which one is intrinsics free.
    - igouy 2y ago
      > language vs. language Where there are few enough programs that readers should check that the programs they compare seem appropriate for their purpose.
      - lossolo 2y ago
        Well, it's your site, so you can do what you want with it, but I don't believe what you just wrote is logical at all. Sometimes, you just want to see, in general, how one language compares to another when one uses intrinsics and the other doesn't, without having to click through every single benchmark across multiple versions to find one without intrinsics. This is just bad UX and waste of time.