4 ms·
I'm just going to put this out there: this benchmark is silly, and only good for making languages with specific types of calling styles look good. It doesn't pr
by KirinDave 8y ago
I'm just going to put this out there: this benchmark is silly, and only good for making languages with specific types of calling styles look good. It doesn't predict any real world performance. It doesn't speak to real world optimizer outcomes. It really just measures how shallow your call abstraction is. Several languages here perform much worse than equivalent code because the way the language interprets and executes naive recursion is different.
I've seen this game before, so the very first place I looked was the Haskell version. Sure enough, it doesn't even try to force the call graph at all. There's even a section entitled, "breaks the benchmark" and it doesn't note Haskell or other languages with different call semantics are not doing the same thing at all. It just says Haskell doesn't terminate.
Confusingly, there is then a "mem" benchmark which seems to ask "What happens if we do this with even the vaguest damn about algorithmic complexity?" But these are so all-over-the-map they don't even come close to measuring the same thing and have different memory usage.
What's frustrating about this is that the double-uncached is always the wrong way to write this code. It's never good, it really doesn't benchmark anything real world. It's not even a very good compiler benchmark because many optimizers actually go under the hood and rewrite code that is in this obvious style to be something else entirely, just to do better on these benchmarks.
For the Haskell benchmark as an example, the best way to to write this lazily is to write a function that consumes and drops a list like so:
fibF n = head $ drop n fibs
where
fibs = 0 : 1 : next fibs
next (first : rest) = (first + head rest) : next rest
-- Or using a library function most Haskell Fp devs know.
fibZip :: Int -> Int
fibZip n = head $ drop n fibz
where fibz = 0 : 1 : zipWith (+) fibz (tail fibz)
This is fast (it gets fused down to essentially a for loop in Haskell, and others like Javascript & Clojure can use this technique for a memory-efficient approach) it's very straightforward, and it's also completely outside the world of something you can write naturally in C or Java (there is a natural Golang expression, but I don't see people using it).
This approach isn't even particularly fair to other languages that are good at producing performant machine code, like Rust.
These games are not really indicative of anything. They waste energy. Folks should understand what their language runtimes and compilers are capable of, rather than asking, "How well does this fare on the worst possible algorithm for an operation who's semantics are only loosely defined?"