3 ms·

the HTTP benchmark is not fair since the crystal implementation is setting the content type explicitly while the Go implementation is auto detecting it.

by dcu 6y ago

Thaxll 6y ago
Indeed the benchmark was dismissed on Reddit coupe of days ago: https://www.reddit.com/r/programming/comments/h0knmi/go_vs_crystal_performance/ https://www.reddit.com/r/programming/comments/h0knmi/go_vs_c...
- dgb23 6y ago
  The repost[0] in r/golang shows similar critiques. This perf comparison is naive on many levels. - binary size (static vs. dynamic linking) - recursion is not idiomatic in go, it is assumed that you write imperative for loops. - mathematical functions like the Fibonacci sequence are a rather atypical computation use-case for a Go program (I don't know about Crystal). Tree/graph traversal/mutations would be a more fitting test. Or generally something that is composed of dynamically growing and shrinking slices and maps. - http test apparently cannot be reproduced, some get better results for Go, some for Crystal. - http tests w/o involving some parsing/marshaling/serialization or something along those lines aren't that useful. You usually want to either read or send some JSON string or similar. [0] https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crystal_performance/ https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crysta...
  - yxhuvud 6y ago
    Recursion is really not the idiomatic way to solve most things in Crystal either, not that it make the test relevant.
- coldtea 6y ago
  That said, "was dismissed on Reddit", is one of the less convincing arguments
  - hombre_fatal 6y ago
    Well, it expands into "was dismissed elsewhere, so go there to see if the arguments there are convincing."
jerf 6y ago
Cards on the table: I'd expect Crystal to outpace Go in general. Go is a "fast language" on the general landscape, but as compiled languages go, it's on the slower side. So this isn't a partisan-driven response. (I'm broadly in favor of the entire landscape of newer languages and look forward to the success of many of them and more.) That said, these two benchmarks straddle both sides of "not very useful"; the first is more likely testing function calls than any sort of what we usually consider "performance", and the benchmark times suggest to me that what you're seeing there is a one or two instruction/cycle difference in the function boilerplate between the two languages. In the vast majority programs, function call time is far from your biggest problem; usually you're writing functions that are much, much larger than the function call overhead. On the other side, as a language benchmark, testing the entire HTTP server stack is way too big. There's just way too many software engineering decisions involved in writing an HTTP server around performance vs. correctness for that to be a fair language test. The autodetection of content type may be the dominant factor today, but it's only an example of a whole class of decisions that can be legitimately made in various ways, plus all the other software from OS on up that you're also benchmarking implicitly. (I'm emphasizing "language" because benchmarking "the simplest, fastest request this web environment can give me" is a useful bit of information. I think it's often greatly overemphasized. Once time-per-request (which I think is a better way to think of it usually rather that requests-per-time) is substantially less than your customer handlers, it stops mattering whether it's .1% of your web request or .06% of your web request time. It's a useful bit of information, but it's only a small bit of information about the web stack in a large pile of other considerations. But it's not a great benchmark of the language's performance because of the aforementioned substantial, legitimate differences in structure and safety/performance tradeoffs that can be made that can dominate the language differences.)

the HTTP benchmark is not fair since the crystal implementation is setting the content type explicitly while the Go implementation is auto detecting it.

by dcu 6y ago

the HTTP benchmark is not fair since the crystal implementation is setting the content type explicitly while the Go implementation is auto detecting it.

Thaxll 6y ago

Indeed the benchmark was dismissed on Reddit coupe of days ago: https://www.reddit.com/r/programming/comments/h0knmi/go_vs_crystal_performance/ https://www.reddit.com/r/programming/comments/h0knmi/go_vs_c...

dgb23 6y ago
The repost[0] in r/golang shows similar critiques. This perf comparison is naive on many levels. - binary size (static vs. dynamic linking) - recursion is not idiomatic in go, it is assumed that you write imperative for loops. - mathematical functions like the Fibonacci sequence are a rather atypical computation use-case for a Go program (I don't know about Crystal). Tree/graph traversal/mutations would be a more fitting test. Or generally something that is composed of dynamically growing and shrinking slices and maps. - http test apparently cannot be reproduced, some get better results for Go, some for Crystal. - http tests w/o involving some parsing/marshaling/serialization or something along those lines aren't that useful. You usually want to either read or send some JSON string or similar. [0] https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crystal_performance/ https://www.reddit.com/r/golang/comments/h0kogq/go_vs_crysta...
- yxhuvud 6y ago
  Recursion is really not the idiomatic way to solve most things in Crystal either, not that it make the test relevant.
coldtea 6y ago
That said, "was dismissed on Reddit", is one of the less convincing arguments
- hombre_fatal 6y ago
  Well, it expands into "was dismissed elsewhere, so go there to see if the arguments there are convincing."

jerf 6y ago

Cards on the table: I'd expect Crystal to outpace Go in general. Go is a "fast language" on the general landscape, but as compiled languages go, it's on the slower side. So this isn't a partisan-driven response. (I'm broadly in favor of the entire landscape of newer languages and look forward to the success of many of them and more.) That said, these two benchmarks straddle both sides of "not very useful"; the first is more likely testing function calls than any sort of what we usually consider "performance", and the benchmark times suggest to me that what you're seeing there is a one or two instruction/cycle difference in the function boilerplate between the two languages. In the vast majority programs, function call time is far from your biggest problem; usually you're writing functions that are much, much larger than the function call overhead. On the other side, as a language benchmark, testing the entire HTTP server stack is way too big. There's just way too many software engineering decisions involved in writing an HTTP server around performance vs. correctness for that to be a fair language test. The autodetection of content type may be the dominant factor today, but it's only an example of a whole class of decisions that can be legitimately made in various ways, plus all the other software from OS on up that you're also benchmarking implicitly. (I'm emphasizing "language" because benchmarking "the simplest, fastest request this web environment can give me" is a useful bit of information. I think it's often greatly overemphasized. Once time-per-request (which I think is a better way to think of it usually rather that requests-per-time) is substantially less than your customer handlers, it stops mattering whether it's .1% of your web request or .06% of your web request time. It's a useful bit of information, but it's only a small bit of information about the web stack in a large pile of other considerations. But it's not a great benchmark of the language's performance because of the aforementioned substantial, legitimate differences in structure and safety/performance tradeoffs that can be made that can dominate the language differences.)