4 ms·

The recursively written Fibonacci benchmark is terrible for comparing a interpreted language to a compiled language. When compiling with optimizations in C, for

The recursively written Fibonacci benchmark is terrible for comparing a interpreted language to a compiled language. When compiling with optimizations in C, for example, the compiler significantly transforms the program. [1] It's possible Crystal does too. [1] https://godbolt.org/z/vaESBG https://godbolt.org/z/vaESBG

eeereerews 6y ago
Recursive fibonacci is highly artificial, but the fact that a compiler can greatly transform the program is an important advantage of compilers over interpreters.
- zucker42 6y ago
  I had the (mistaken) impression that the GCC code signicantly transformed the algorithm because there is only one recursive call instruction, but looking at it closer it's still exponential. I still think it's a pretty silly idea to use such an unrealistic benchmark. Especially since it might be possible to make a compiler turn the exponential algorithm linear.
  - nv-vn 6y ago
    I'd be very surprised if there's a commercial compiler that would automatically optimize this into linear time. Maybe some obscure functional research language? But certainly don't think LLVM is capable of dynamic programming.
- ken 6y ago
  That's sort of true, in the abstract, but it wouldn't explain what's going on here. Even JavaScript (with a JIT, but not a separate compilation step, and certainly no type annotations) is faster at this than compiled Crystal. There's clearly other factors which are much more significant than simply having a compiler.
- mumblemumble 6y ago
  When you're doing something like writing your own routine for calculating fibonacci numbers, perhaps. But that's not the kind of optimization that's likely to be significant to most developers writing business applications. I would also argue that the ubiquity of things like BLAS wrappers largely blurs these lines. For example, one of the big reasons I choose Python over Java (my company's primary language) for my work is that, thanks to numpy, Python absolutely smokes Java at crunching numbers, for my purposes. This despite Java being a compiled static language and Python being an interpreted dynamic language.
bjoli 6y ago
It can be an ok way to measure procedure call overhead though (between languages with similar semantics)
- ken 6y ago
  Maybe, but these two languages don't exactly have similar semantics. You can't make a Ruby program that uses fixnums-only. You also can't make a Crystal program that's dynamic like Ruby. Interestingly, though, when the Crystal implementation is modified to use bignums, the Ruby implementation is over twice as fast while still being dynamic. What I get from this is that Crystal's static compiler is slower than Ruby's dynamically dispatched interpreter, and Crystal's bignums are so slow you'll need to think hard about whether you want a 50x speed boost or correct arithmetic in all cases. I think Crystal is a good concept, but it's only version 0.34.0. Every implementation is bad at version <<1.0. Ruby <<1.0 didn't have good performance, either. I'm sure Crystal will be great by the time it gets to 2.6.5, too.

eeereerews 6y ago

Recursive fibonacci is highly artificial, but the fact that a compiler can greatly transform the program is an important advantage of compilers over interpreters.

zucker42 6y ago
I had the (mistaken) impression that the GCC code signicantly transformed the algorithm because there is only one recursive call instruction, but looking at it closer it's still exponential. I still think it's a pretty silly idea to use such an unrealistic benchmark. Especially since it might be possible to make a compiler turn the exponential algorithm linear.
- nv-vn 6y ago
  I'd be very surprised if there's a commercial compiler that would automatically optimize this into linear time. Maybe some obscure functional research language? But certainly don't think LLVM is capable of dynamic programming.
ken 6y ago
That's sort of true, in the abstract, but it wouldn't explain what's going on here. Even JavaScript (with a JIT, but not a separate compilation step, and certainly no type annotations) is faster at this than compiled Crystal. There's clearly other factors which are much more significant than simply having a compiler.
mumblemumble 6y ago
When you're doing something like writing your own routine for calculating fibonacci numbers, perhaps. But that's not the kind of optimization that's likely to be significant to most developers writing business applications. I would also argue that the ubiquity of things like BLAS wrappers largely blurs these lines. For example, one of the big reasons I choose Python over Java (my company's primary language) for my work is that, thanks to numpy, Python absolutely smokes Java at crunching numbers, for my purposes. This despite Java being a compiled static language and Python being an interpreted dynamic language.

bjoli 6y ago

It can be an ok way to measure procedure call overhead though (between languages with similar semantics)

ken 6y ago
Maybe, but these two languages don't exactly have similar semantics. You can't make a Ruby program that uses fixnums-only. You also can't make a Crystal program that's dynamic like Ruby. Interestingly, though, when the Crystal implementation is modified to use bignums, the Ruby implementation is over twice as fast while still being dynamic. What I get from this is that Crystal's static compiler is slower than Ruby's dynamically dispatched interpreter, and Crystal's bignums are so slow you'll need to think hard about whether you want a 50x speed boost or correct arithmetic in all cases. I think Crystal is a good concept, but it's only version 0.34.0. Every implementation is bad at version <<1.0. Ruby <<1.0 didn't have good performance, either. I'm sure Crystal will be great by the time it gets to 2.6.5, too.