13 ms·

Looking at the reverse-complement code, it appears that the Rust and C implementations are using different algorithms: https://benchmarksgame-team.pages.debian

Looking at the reverse-complement code, it appears that the Rust and C implementations are using different algorithms: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-rust-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-6.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... On a quick inspection: - The Rust code is about twice as long. - The Rust code has CPU feature detection and SSE intrinsics, while the C code is more idiomatic. - The lookup table is larger in the Rust code.

vitus 6y ago
I recall an anecdote about how Haskell actually outperformed C on various tree benchmarks because it was using a better implementation. At some point, the C programmers got fed up with the airs of superiority from Haskell programmers, ported the Haskell implementation, and reclaimed their position. I wouldn't be surprised if there's something similar happening here.
- _wldu 6y ago
  Yes, it seems everyone is always trying to beat C, but really, no one can.
  - lawn 6y ago
    To be fair, if there's a language that has a fair chance it's Rust.
  - pjmlp 6y ago
    Well, junior Assembly developers had it pretty easy on 80's hardware. Also C++ and Fortran don't have to spend much effort to achieve that.
- harporoeder 6y ago
  There was a long period where a fairly unknown theorem proving language ATS (1) was beating C in many test cases on the benchmark game (2) the benchmarks were removed though (3). I expect many languages could be made to win with sufficient effort. 1. http://www.ats-lang.org/ http://www.ats-lang.org/ 2. http://web.archive.org/web/20121218042116/http://shootout.alioth.debian.org/u64/ats.php http://web.archive.org/web/20121218042116/http://shootout.al... 3. https://stackoverflow.com/questions/26958969/why-was-the-ats-language-dropped-from-the-computer-language-benchmarks-game https://stackoverflow.com/questions/26958969/why-was-the-ats...
  - steinuil 6y ago
    ATS is not really a theorem proving language, it's almost a superset of C with a very long list of type system and language features that lets you write anything from C code with no safety to very high level recursive functional code with lifetime tracking which will translate to efficient C code, if the transformations are proved to be correct. It's a weird beast, but I'm not surprised it outperformed some C implementations, because it is basically C with a lot more features.
    - ghostwriter 6y ago
      > ATS is not really a theorem proving language What is "real theorem-proving language" in this context? ATS has ATS/LF that is designed for writing formal proofs in the language. http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/HTMLTOC/c2867.html http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...
- spiffytech 6y ago
  That kind of tit-for-tat in benchmarks seems like it's counter to the goal of benchmarks: what kind of performance could I expect to see using technology $FOO? Crucially, that question depends on how someone will realistically implement $FOO. I like PyPy as an example: on the surface, implementing a Python runtime in Python and expecting performance gains seems crazy. PyPy manages to outperform CPython because although a C implementation should theoretically be faster, realistically the increased expressiveness of Python lets the PyPy devs opt into optimizations the CPython devs find out of reach. I don't know C or Rust well enough to comment on these specific scenarios, but if two technologies can be fast, and one makes that speed accessible while the other encourages implementors to leave performance on the table, that's much more useful information to me than seeing a back-and-forth where folks compete to implement optimizations I'll never make in my own projects.
  - vitus 6y ago
    You raise a good point. I've always been fascinating by PyPy's performance, personally -- anecdotally, I've achieved ~10x speedups from just running a script with `pypy` instead of `python`. I always attributed that to better performance of the JIT, but I could be wrong. I have nothing against Rust personally, but it's ultimately not an apples-to-apples comparison if they're not implementing the same algorithm, or even if they're not using the same mechanisms (e.g. Rust explicitly using SSE intrinsics, which are certainly available to C in just as idiomatic a fashion).
    - gameswithgo 6y ago
      comparing apples to apples is pointless, they are both apples. and you can always compare “equivalent” algorithms as some languages may not be able to efficiently express the same algorithm as another. i know what you are after, but trying to have some benchmark that is “fair” according to some spec that is important to your needs will just be seen as pointless to others. benchmark game at least lets us see what the performance ceiling is of various languages, and anytime you object to a language having a worse implementation you may go fix it. i think what anyone with a clue understands is that C C++ and Rust all have roughly equivalent performance ceilings. Nobody really thinks Rust is faster than C now.
- loxias 6y ago
  It's exactly the same thing. I've been seeing this tossed around for years now, and one of these days it'll make me grumpy enough to fix the benchmark.
- Blikkentrekker 6y ago
  The Haskell code on some pathological examples of these implementations that I have seen has so many unsafe construct, strictness annotations, inlining annotations and so forth that it's practically C in a different syntax. It is not idiomatic Haskell at all and loses all of the touted benefits. Is there a separate benchmark that only accepts idiomatic code?
  - pjmlp 6y ago
    Just like C is pimped up BCPL, a language designed to bootstrap the CPL compiler, 10 years younger than ESPOL/NEWP, 10 years younger than Jovial, 6 years younger than PL/I, among other possible examples from the 60's. Also C was known for not being a fast language on 80's home hardware, ruled by Assembly. So low level coding for OS isn't coding like C and is just the way for mechanical sympathy, regardless of the language.
  - igouy 6y ago
    Is there a GHC compiler switch that ensures only "idiomatic code" is compiled? One can, with sufficient effort, essentially write C code in Haskell using various unsafe primitives. We would argue that this is not true to the spirit and goals of Haskell, and we have attempted in this paper to remain within the space of “reasonably idiomatic” Haskell. However, we have made abundant use of strictness annotations, explicit strictness, and unboxed vectors. We have, more controversially perhaps, used unsafe array subscripting in places. Are our choices reasonable? http://www.cs.ru.nl/P.Achten/IFL2013/symposium_proceedings_IFL2013/ifl2013_submission_20.pdf http://www.cs.ru.nl/P.Achten/IFL2013/symposium_proceedings_I...
    - Blikkentrekker 6y ago
      It shows the performance of that variant of Haskell. But when using that variant all the correctness guarantees that Haskell is known for are no longer on the table.
- FartyMcFarter 6y ago
  > I wouldn't be surprised if there's something similar happening here. In this case it seems like benchmark code is allowed to use intrinsics, which can degenerate into a situation where a benchmark in language X is more "glorified x86 Assembly code" than actual code in language X. This is not very useful for comparing languages IMO. Especially since all of Rust, C, C++ can use this strategy and become almost identical in both code and performance.
  - gameswithgo 6y ago
    So three languages with identical performance ceilings end up with essentially identical performance results. sounds like a job well done. if you want to know something more than that, like how performance tends to end up after inputs of identical effort and talent you will need to recruit a lot of people and money to run really large scale experiments and when you are done people on HN will just endlessly find nits to pick with your results whenever they don’t agree with their preconceived notions
- madacol 6y ago
  Looked at the git repo https://salsa.debian.org/benchmarksgame-team/benchmarksgame https://salsa.debian.org/benchmarksgame-team/benchmarksgame and it seems there's only one committer. So if all scripts are written by the same guy, then his biases are all over the place, and it needs more code diversity before we can arrive to any conclusion. Am I missing something?, is there a list somewhere of the people who have contributed to this?
  - Narishma 6y ago
    Click on any of the benchmark programs and it will have the contributors' names at the beginning.
    - madacol 6y ago
      Thanks!
- johnisgood 6y ago
  It is always like that. There is no way that X is faster than C. It most definitely has to do with algorithm. What I can believe is Fortran being faster than C in some circumstances, but that is about it. Can you guys please stop this nonsense? No, your equivalent Rust code is not faster, sorry.
pornel 6y ago
Nothing stops someone from copying and submitting other implementation's algorithm. There are multiple implementations of each benchmark for every language: • https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-rust-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-rust-2.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-rust-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... It's possible that someone has already submitted both algorithms for both languages, and different approaches won for language-specific reasons.
- FartyMcFarter 6y ago
  These are all the C versions I can find: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-2.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-4.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-6.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... None of them have SSE intrinsics or are quite as long as the Rust version. I find it doubtful that SSE intrinsics wouldn't help the C version, if they are indeed helping the Rust version. This seems fairly easy to check as the Rust version has a non-SSE fallback code path - I'd do it myself but am not able to at the moment.
- littlestymaar 6y ago
  > Nothing stops someone Well, it looks like the submission process[0] and the maintainer[1] do, actually. [0]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_overall_faster_than_c_in_benchmarks/ghzpo7h/?utm_source=reddit&utm_medium=web2x&context=3 https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov... [1]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_overall_faster_than_c_in_benchmarks/ghzyb9g/?utm_source=reddit&utm_medium=web2x&context=3 https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov...
mhh__ 6y ago
> The Rust code has CPU feature detection and SSE intrinsics, while the C code is more idiomatic. These benchmarks are almost always either useless or a scam - you either end up writing rewriting the same implementation n times or you don't utilize the capabilities of the language, either way you're not really measuring much of anything intrinsic to the language itself - Rust and C both have the same backends and if you really care about performance you're going to take it to the max anyway, so inference by the compiler isn't that important.
gameswithgo 6y ago
cpu feature detection is way easier in rust. its just built into the core libs.
- colejohnson66 6y ago
  Doesn’t gcc have a CPU feature branching system? You use it by attaching attributes to functions that say what instructions are required. Granted, it’s not as “elegant” as Rust, but it does exist.
- bsder 6y ago
  And? These are the kind of things that make working in one language different from another. I'd actually like to see the SSE version in C so I can compare the two implementations and how much grief you have to go through.
qart 6y ago
I have seen this happening so often: C/C++/Rust often end up using CPU-specific features, and the code starts looking more and more like assembly code, and less like idiomatic high-level language code. Basically, comparisons of programs written in all the other languages against these three become meaningless. And in turn, hurts benchmarksgame as a resource for comparing languages. If I had to write a performant library at work, I too might rely on CPU-specific assembly wrappers in my code. But IMO, such code has no place in a general-purpose cross-language benchmark site.
- igouy 6y ago
  > If I had to write a performant library at work, I too might rely on CPU-specific assembly wrappers in my code. But IMO, such code has no place in a general-purpose cross-language benchmark site. My guess is that other people would take your "performant library at work" premise as justification for including that code.
fsloth 6y ago
What does "idiomatic" C even mean? It's a high level assembler and as such should not limit the creativity of programmers using it. C code that pretends it does not need to care about it's platform is not idiomatic, it's just suboptimal.
- FartyMcFarter 6y ago
  By "idiomatic C" I meant any of the following: - Code that most C books/courses would teach you how to write - Portable C code (arguably portability is one of C's biggest successes!) - Code that you'd expect to find in the K&R book
  - pjmlp 6y ago
    One of C's biggest marketing successes. All high level languages are portable, including some older than C. It helps when one drags the OS the language was born on, along via parallel standards like POSIX.
    - FartyMcFarter 6y ago
      > All high level languages are portable, including some older than C. Theoretically portable, maybe. But C is actually portable in practice - you can compile C code for the vast majority of CPUs and OSs.
      - Someone 6y ago
        Devil’s advocate: compared to assembly, C is portable, but compared to a lot of high-level languages, C isn’t very portable. Every “implementation defined” feature (https://en.wikipedia.org/wiki/Unspecified_behavior#Implementation-defined_behavior https://en.wikipedia.org/wiki/Unspecified_behavior#Implement...) creates two different languages (examples are sizes of basic numeric types, signedness of the char type, and evaluation order of arguments) Also, the standard library is so limited that, historically, you needed lots of stuff that wasn’t standardized between systems (https://en.wikipedia.org/wiki/GNU_Autotools https://en.wikipedia.org/wiki/GNU_Autotools: “It can be difficult to make a software program portable: the C compiler differs from system to system; certain library functions are missing on some systems; header files may have different names.”) But yes, bringing up C on a system is a lot easier than doing the same for, to pick another extreme, Ada, but that’s because only half the job is done. Autotools does a bit of the rest, but still requires help from program writers.
JZerf 6y ago
I'm the author of that C program. You are correct that the programs are using somewhat different algorithms (and as an author for many other submissions on the site, I can say this is true for many of the benchmarks on the site). In addition to what you've already mentioned, if I'm not mistaken, the programs also take different multi-threading approaches. One of the tasks in this benchmark involves reversing three very large strings. My C program took a fairly simple approach and does that reversal process for each string serially but processes multiple strings in parallel. The Rust program on the other hand does the opposite and does the reversal process for each string in a parallel manner but only processes strings one at a time. The algorithm the Rust program uses is more complicated which results in a considerably larger program size that also is a bit less flexible regarding the input (it assumes each line of input is 60 characters whereas my program will work with lines of any length) but it results in better CPU utilization, less clock time, and less memory use. I've been meaning to submit a faster C program using this faster algorithm but have simply been too busy with other things.

vitus 6y ago

I recall an anecdote about how Haskell actually outperformed C on various tree benchmarks because it was using a better implementation. At some point, the C programmers got fed up with the airs of superiority from Haskell programmers, ported the Haskell implementation, and reclaimed their position. I wouldn't be surprised if there's something similar happening here.

_wldu 6y ago
Yes, it seems everyone is always trying to beat C, but really, no one can.
- lawn 6y ago
  To be fair, if there's a language that has a fair chance it's Rust.
- pjmlp 6y ago
  Well, junior Assembly developers had it pretty easy on 80's hardware. Also C++ and Fortran don't have to spend much effort to achieve that.
harporoeder 6y ago
There was a long period where a fairly unknown theorem proving language ATS (1) was beating C in many test cases on the benchmark game (2) the benchmarks were removed though (3). I expect many languages could be made to win with sufficient effort. 1. http://www.ats-lang.org/ http://www.ats-lang.org/ 2. http://web.archive.org/web/20121218042116/http://shootout.alioth.debian.org/u64/ats.php http://web.archive.org/web/20121218042116/http://shootout.al... 3. https://stackoverflow.com/questions/26958969/why-was-the-ats-language-dropped-from-the-computer-language-benchmarks-game https://stackoverflow.com/questions/26958969/why-was-the-ats...
- steinuil 6y ago
  ATS is not really a theorem proving language, it's almost a superset of C with a very long list of type system and language features that lets you write anything from C code with no safety to very high level recursive functional code with lifetime tracking which will translate to efficient C code, if the transformations are proved to be correct. It's a weird beast, but I'm not surprised it outperformed some C implementations, because it is basically C with a lot more features.
  - ghostwriter 6y ago
    > ATS is not really a theorem proving language What is "real theorem-proving language" in this context? ATS has ATS/LF that is designed for writing formal proofs in the language. http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/HTMLTOC/c2867.html http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...
spiffytech 6y ago
That kind of tit-for-tat in benchmarks seems like it's counter to the goal of benchmarks: what kind of performance could I expect to see using technology $FOO? Crucially, that question depends on how someone will realistically implement $FOO. I like PyPy as an example: on the surface, implementing a Python runtime in Python and expecting performance gains seems crazy. PyPy manages to outperform CPython because although a C implementation should theoretically be faster, realistically the increased expressiveness of Python lets the PyPy devs opt into optimizations the CPython devs find out of reach. I don't know C or Rust well enough to comment on these specific scenarios, but if two technologies can be fast, and one makes that speed accessible while the other encourages implementors to leave performance on the table, that's much more useful information to me than seeing a back-and-forth where folks compete to implement optimizations I'll never make in my own projects.
- vitus 6y ago
  You raise a good point. I've always been fascinating by PyPy's performance, personally -- anecdotally, I've achieved ~10x speedups from just running a script with `pypy` instead of `python`. I always attributed that to better performance of the JIT, but I could be wrong. I have nothing against Rust personally, but it's ultimately not an apples-to-apples comparison if they're not implementing the same algorithm, or even if they're not using the same mechanisms (e.g. Rust explicitly using SSE intrinsics, which are certainly available to C in just as idiomatic a fashion).
  - gameswithgo 6y ago
    comparing apples to apples is pointless, they are both apples. and you can always compare “equivalent” algorithms as some languages may not be able to efficiently express the same algorithm as another. i know what you are after, but trying to have some benchmark that is “fair” according to some spec that is important to your needs will just be seen as pointless to others. benchmark game at least lets us see what the performance ceiling is of various languages, and anytime you object to a language having a worse implementation you may go fix it. i think what anyone with a clue understands is that C C++ and Rust all have roughly equivalent performance ceilings. Nobody really thinks Rust is faster than C now.
loxias 6y ago
It's exactly the same thing. I've been seeing this tossed around for years now, and one of these days it'll make me grumpy enough to fix the benchmark.
Blikkentrekker 6y ago
The Haskell code on some pathological examples of these implementations that I have seen has so many unsafe construct, strictness annotations, inlining annotations and so forth that it's practically C in a different syntax. It is not idiomatic Haskell at all and loses all of the touted benefits. Is there a separate benchmark that only accepts idiomatic code?
- pjmlp 6y ago
  Just like C is pimped up BCPL, a language designed to bootstrap the CPL compiler, 10 years younger than ESPOL/NEWP, 10 years younger than Jovial, 6 years younger than PL/I, among other possible examples from the 60's. Also C was known for not being a fast language on 80's home hardware, ruled by Assembly. So low level coding for OS isn't coding like C and is just the way for mechanical sympathy, regardless of the language.
- igouy 6y ago
  Is there a GHC compiler switch that ensures only "idiomatic code" is compiled? One can, with sufficient effort, essentially write C code in Haskell using various unsafe primitives. We would argue that this is not true to the spirit and goals of Haskell, and we have attempted in this paper to remain within the space of “reasonably idiomatic” Haskell. However, we have made abundant use of strictness annotations, explicit strictness, and unboxed vectors. We have, more controversially perhaps, used unsafe array subscripting in places. Are our choices reasonable? http://www.cs.ru.nl/P.Achten/IFL2013/symposium_proceedings_IFL2013/ifl2013_submission_20.pdf http://www.cs.ru.nl/P.Achten/IFL2013/symposium_proceedings_I...
  - Blikkentrekker 6y ago
    It shows the performance of that variant of Haskell. But when using that variant all the correctness guarantees that Haskell is known for are no longer on the table.
FartyMcFarter 6y ago
> I wouldn't be surprised if there's something similar happening here. In this case it seems like benchmark code is allowed to use intrinsics, which can degenerate into a situation where a benchmark in language X is more "glorified x86 Assembly code" than actual code in language X. This is not very useful for comparing languages IMO. Especially since all of Rust, C, C++ can use this strategy and become almost identical in both code and performance.
- gameswithgo 6y ago
  So three languages with identical performance ceilings end up with essentially identical performance results. sounds like a job well done. if you want to know something more than that, like how performance tends to end up after inputs of identical effort and talent you will need to recruit a lot of people and money to run really large scale experiments and when you are done people on HN will just endlessly find nits to pick with your results whenever they don’t agree with their preconceived notions
madacol 6y ago
Looked at the git repo https://salsa.debian.org/benchmarksgame-team/benchmarksgame https://salsa.debian.org/benchmarksgame-team/benchmarksgame and it seems there's only one committer. So if all scripts are written by the same guy, then his biases are all over the place, and it needs more code diversity before we can arrive to any conclusion. Am I missing something?, is there a list somewhere of the people who have contributed to this?
- Narishma 6y ago
  Click on any of the benchmark programs and it will have the contributors' names at the beginning.
  - madacol 6y ago
    Thanks!
johnisgood 6y ago
It is always like that. There is no way that X is faster than C. It most definitely has to do with algorithm. What I can believe is Fortran being faster than C in some circumstances, but that is about it. Can you guys please stop this nonsense? No, your equivalent Rust code is not faster, sorry.

pornel 6y ago

Nothing stops someone from copying and submitting other implementation's algorithm. There are multiple implementations of each benchmark for every language: • https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-rust-1.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-rust-2.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... • https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-rust-3.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... It's possible that someone has already submitted both algorithms for both languages, and different approaches won for language-specific reasons.

FartyMcFarter 6y ago
These are all the C versions I can find: https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-2.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-4.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-5.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/revcomp-gcc-6.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... None of them have SSE intrinsics or are quite as long as the Rust version. I find it doubtful that SSE intrinsics wouldn't help the C version, if they are indeed helping the Rust version. This seems fairly easy to check as the Rust version has a non-SSE fallback code path - I'd do it myself but am not able to at the moment.
littlestymaar 6y ago
> Nothing stops someone Well, it looks like the submission process[0] and the maintainer[1] do, actually. [0]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_overall_faster_than_c_in_benchmarks/ghzpo7h/?utm_source=reddit&utm_medium=web2x&context=3 https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov... [1]: https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_overall_faster_than_c_in_benchmarks/ghzyb9g/?utm_source=reddit&utm_medium=web2x&context=3 https://www.reddit.com/r/rust/comments/kpqmrh/rust_is_now_ov...

mhh__ 6y ago

> The Rust code has CPU feature detection and SSE intrinsics, while the C code is more idiomatic. These benchmarks are almost always either useless or a scam - you either end up writing rewriting the same implementation n times or you don't utilize the capabilities of the language, either way you're not really measuring much of anything intrinsic to the language itself - Rust and C both have the same backends and if you really care about performance you're going to take it to the max anyway, so inference by the compiler isn't that important.

gameswithgo 6y ago

cpu feature detection is way easier in rust. its just built into the core libs.

colejohnson66 6y ago
Doesn’t gcc have a CPU feature branching system? You use it by attaching attributes to functions that say what instructions are required. Granted, it’s not as “elegant” as Rust, but it does exist.
bsder 6y ago
And? These are the kind of things that make working in one language different from another. I'd actually like to see the SSE version in C so I can compare the two implementations and how much grief you have to go through.

qart 6y ago

I have seen this happening so often: C/C++/Rust often end up using CPU-specific features, and the code starts looking more and more like assembly code, and less like idiomatic high-level language code. Basically, comparisons of programs written in all the other languages against these three become meaningless. And in turn, hurts benchmarksgame as a resource for comparing languages. If I had to write a performant library at work, I too might rely on CPU-specific assembly wrappers in my code. But IMO, such code has no place in a general-purpose cross-language benchmark site.

igouy 6y ago
> If I had to write a performant library at work, I too might rely on CPU-specific assembly wrappers in my code. But IMO, such code has no place in a general-purpose cross-language benchmark site. My guess is that other people would take your "performant library at work" premise as justification for including that code.

fsloth 6y ago

What does "idiomatic" C even mean? It's a high level assembler and as such should not limit the creativity of programmers using it. C code that pretends it does not need to care about it's platform is not idiomatic, it's just suboptimal.

FartyMcFarter 6y ago
By "idiomatic C" I meant any of the following: - Code that most C books/courses would teach you how to write - Portable C code (arguably portability is one of C's biggest successes!) - Code that you'd expect to find in the K&R book
- pjmlp 6y ago
  One of C's biggest marketing successes. All high level languages are portable, including some older than C. It helps when one drags the OS the language was born on, along via parallel standards like POSIX.
  - FartyMcFarter 6y ago
    > All high level languages are portable, including some older than C. Theoretically portable, maybe. But C is actually portable in practice - you can compile C code for the vast majority of CPUs and OSs.
    - Someone 6y ago
      Devil’s advocate: compared to assembly, C is portable, but compared to a lot of high-level languages, C isn’t very portable. Every “implementation defined” feature (https://en.wikipedia.org/wiki/Unspecified_behavior#Implementation-defined_behavior https://en.wikipedia.org/wiki/Unspecified_behavior#Implement...) creates two different languages (examples are sizes of basic numeric types, signedness of the char type, and evaluation order of arguments) Also, the standard library is so limited that, historically, you needed lots of stuff that wasn’t standardized between systems (https://en.wikipedia.org/wiki/GNU_Autotools https://en.wikipedia.org/wiki/GNU_Autotools: “It can be difficult to make a software program portable: the C compiler differs from system to system; certain library functions are missing on some systems; header files may have different names.”) But yes, bringing up C on a system is a lot easier than doing the same for, to pick another extreme, Ada, but that’s because only half the job is done. Autotools does a bit of the rest, but still requires help from program writers.

JZerf 6y ago

I'm the author of that C program. You are correct that the programs are using somewhat different algorithms (and as an author for many other submissions on the site, I can say this is true for many of the benchmarks on the site). In addition to what you've already mentioned, if I'm not mistaken, the programs also take different multi-threading approaches. One of the tasks in this benchmark involves reversing three very large strings. My C program took a fairly simple approach and does that reversal process for each string serially but processes multiple strings in parallel. The Rust program on the other hand does the opposite and does the reversal process for each string in a parallel manner but only processes strings one at a time. The algorithm the Rust program uses is more complicated which results in a considerably larger program size that also is a bit less flexible regarding the input (it assumes each line of input is 60 characters whereas my program will work with lines of any length) but it results in better CPU utilization, less clock time, and less memory use. I've been meaning to submit a faster C program using this faster algorithm but have simply been too busy with other things.