23 ms·

Consider the fasta benchmark. It would be completely disingenuous to suggest that Haskell is on top for any reason other than it doing a different thing to the

by Veedrac 11y ago

Consider the fasta benchmark. It would be completely disingenuous to suggest that Haskell is on top for any reason other than it doing a different thing to the other benchmarks. Haskell is just not that fast. So, in a sense, it cheats. Of course, I copied these techniques in my Rust implementation. So, if Haskell cheated, I cheated. The same can be said for almost all other benchmarks - but not normally quite to this extent. But it's not cheating in a literal sense, since (AFAIK) the Haskell program has been accepted on the basis that it follows the rules. So it doesn't cheat in that it follows the rules, but it cheats in that it should be breaking the rules. I can be more specific if needed.

codygman 11y ago
> I can be more specific if needed. Please do! Can you give some specific examples, preferably in order of those with the most effect on performance.
- Veedrac 11y ago
  Since I'm lazy, I'll point out that most of the stuff is indirectly mentioned in this write-up: https://llogiq.github.io/2015/10/03/fast.html https://llogiq.github.io/2015/10/03/fast.html This reminds me I haven't written up stuff for chameneos-redux, which is probably the most fun and over-engineered of them all. The fasta/Haskell thing I was mentioning are the parts at "The Haskell code does a quite clever alternative" and "the Haskell code just caches them all in an array". The pre-compacting I mention in k_nucleotide was already done in the Scala implementation. How parallelism is added also varies. The Scala code does each of the test cases in parallel whilst the Rust parallelizes each internally. (My new Rust code parallelizes even better.) Several implementation make their own specialized hash table, and there isn't one particular hash table to implement. This is very important since hash table lookups take a substantial fraction of the time. Note that I pick on Haskell and Scala here only because they have the best hacks, not because they're the only ones doing hacks. (I'd imagine listing all of the differences would take much longer than I have time for.) thread-ring has lots of different implementations. Some are just scheduled coroutines restricted to a single process, then you have stuff like [1] which uses mutexes, real threads and sets thread affinity. I'm not totally sure what [2] is but I think it's something else entirely. I'm not being too specific here because honestly this benchmark confuses me. [1] http://benchmarksgame.alioth.debian.org/u64q/program.php?test=threadring&lang=gcc&id=3 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... [2] http://benchmarksgame.alioth.debian.org/u64/program.php?test=threadring&lang=gpp&id=5 http://benchmarksgame.alioth.debian.org/u64/program.php?test...
igouy 11y ago
Please be specific -- Are you saying that the benchmarks game programs you wrote "cheat" ?
- Veedrac 11y ago
  If the Haskell fasta programs cheat, my fasta program cheats. If the Haskell fasta programs do not cheat, my fasta program does not cheat. These statements are made to the best of my knowledge, although someone with a definitive understanding of the rules (you, perhaps) is in a better position to make such decisions. I can make similar comments about my other submissions, referring in each case to different competitors.
  - igouy 11y ago
    You talk about fairness and cheating but you refuse to say unambiguously whether you think your own programs are cheating.
    - Veedrac 11y ago
      I'll be unambiguous when you make the rules unambiguous. I'm being ambiguous because you're forcing me to. I've told you what I'm unsure about. I've linked to write-ups of what I've done. I've pointed to parts in other programs that seem not to clearly match against the rules. Yet you, who knows the rules, have opted not to make one clarification about this. Not a single "yes, that technique is legal" or "no, this is against the rules". If you did this, I would happily give a clearer response. If my programs do break the rules, I would happily retract them until they don't, and point out which other submitted programs should also be retracted. This would not bother me. Instead, you're demanding me to make definitive statements on information you're keeping hidden from me. This is not making me feel pleasant, and this does bother me.
      - igouy 11y ago
        You accuse programs of cheating and then refuse to say what you think they are doing wrong: "Haskell is just not that fast. So, in a sense, it cheats." : that is not "pleasant", it's name-calling. When you say other programs are "cheating-but-not-cheating" : that is not "pleasant", it's name-calling.

by Veedrac 11y ago

codygman 11y ago

> I can be more specific if needed. Please do! Can you give some specific examples, preferably in order of those with the most effect on performance.

Veedrac 11y ago
Since I'm lazy, I'll point out that most of the stuff is indirectly mentioned in this write-up: https://llogiq.github.io/2015/10/03/fast.html https://llogiq.github.io/2015/10/03/fast.html This reminds me I haven't written up stuff for chameneos-redux, which is probably the most fun and over-engineered of them all. The fasta/Haskell thing I was mentioning are the parts at "The Haskell code does a quite clever alternative" and "the Haskell code just caches them all in an array". The pre-compacting I mention in k_nucleotide was already done in the Scala implementation. How parallelism is added also varies. The Scala code does each of the test cases in parallel whilst the Rust parallelizes each internally. (My new Rust code parallelizes even better.) Several implementation make their own specialized hash table, and there isn't one particular hash table to implement. This is very important since hash table lookups take a substantial fraction of the time. Note that I pick on Haskell and Scala here only because they have the best hacks, not because they're the only ones doing hacks. (I'd imagine listing all of the differences would take much longer than I have time for.) thread-ring has lots of different implementations. Some are just scheduled coroutines restricted to a single process, then you have stuff like [1] which uses mutexes, real threads and sets thread affinity. I'm not totally sure what [2] is but I think it's something else entirely. I'm not being too specific here because honestly this benchmark confuses me. [1] http://benchmarksgame.alioth.debian.org/u64q/program.php?test=threadring&lang=gcc&id=3 http://benchmarksgame.alioth.debian.org/u64q/program.php?tes... [2] http://benchmarksgame.alioth.debian.org/u64/program.php?test=threadring&lang=gpp&id=5 http://benchmarksgame.alioth.debian.org/u64/program.php?test...

igouy 11y ago

Please be specific -- Are you saying that the benchmarks game programs you wrote "cheat" ?

Veedrac 11y ago
If the Haskell fasta programs cheat, my fasta program cheats. If the Haskell fasta programs do not cheat, my fasta program does not cheat. These statements are made to the best of my knowledge, although someone with a definitive understanding of the rules (you, perhaps) is in a better position to make such decisions. I can make similar comments about my other submissions, referring in each case to different competitors.
- igouy 11y ago
  You talk about fairness and cheating but you refuse to say unambiguously whether you think your own programs are cheating.
  - Veedrac 11y ago
    I'll be unambiguous when you make the rules unambiguous. I'm being ambiguous because you're forcing me to. I've told you what I'm unsure about. I've linked to write-ups of what I've done. I've pointed to parts in other programs that seem not to clearly match against the rules. Yet you, who knows the rules, have opted not to make one clarification about this. Not a single "yes, that technique is legal" or "no, this is against the rules". If you did this, I would happily give a clearer response. If my programs do break the rules, I would happily retract them until they don't, and point out which other submitted programs should also be retracted. This would not bother me. Instead, you're demanding me to make definitive statements on information you're keeping hidden from me. This is not making me feel pleasant, and this does bother me.
    - igouy 11y ago
      You accuse programs of cheating and then refuse to say what you think they are doing wrong: "Haskell is just not that fast. So, in a sense, it cheats." : that is not "pleasant", it's name-calling. When you say other programs are "cheating-but-not-cheating" : that is not "pleasant", it's name-calling.