4 ms·

I really can't imagine that this benchmark is relevant anymore. Haven't Go, D, and Erlang had major changes since then? E.g., Go is now compiled by a compiler w

by nanny 11y ago

I really can't imagine that this benchmark is relevant anymore. Haven't Go, D, and Erlang had major changes since then? E.g., Go is now compiled by a compiler written in Go.

atilaneves 11y ago
I'm planning on updating it this year once I finally find the time to learn Rust. Developing an MQTT broker has become my go-to task for learning new languages.
- acqq 11y ago
  I'm really glad that you've done the D implementation. I think D gets less attention than it should, especially compared to Go. Maybe somebody already proficient in Rust, reading this, can contribute?
- nanny 11y ago
  No big deal, it wasn't a slight against you, but rather the OP. It's not your fault he reposted your two year old benchmarks.
- pron 11y ago
  How about Java, then? It's a little less esoteric than most of those languages.
  - atilaneves 11y ago
    Funny you should ask: https://atilanevesoncode.wordpress.com/2014/01/08/adding-java-and-c-to-the-mqtt-benchmarks-or-how-i-learned-to-stop-worrying-and-love-the-garbage-collector/ https://atilanevesoncode.wordpress.com/2014/01/08/adding-jav...
    - pron 11y ago
      So Java is the pingtest winner, and I'm sure that with just a little more work it can be the loadtest winner as well.
      - atilaneves 11y ago
        I doubt it, Patrick properly profiled and optimised it. Later on I got the D version to within 3% of Java's performance in pingtest.
        pron 11y ago
        It's clearly optimized for latency over throughput. For one, it is single threaded -- unlike the Erlang, Go and, I believe D (vibe.d) implementations. Also, I don't know exactly how the benchmark was performed, but Java takes time to warm up (otherwise you're testing a completely different implementation of Java, which is only used at startup time). I've found that for long-running server-side apps, Java is very hard to beat in terms of performance, especially relative to the effort spent. I also think it's a nicer language than Go (I also prefer it over C++ and D).
rubiquity 11y ago
How Go is compiled isn't as relevant to its performance as much as what Go is compiled into, which hasn't changed all that much as far as I know. The GC improvements might help with latency though.
- voidlogic 11y ago
  >as much as what Go is compiled into, which hasn't changed all that much as far as I know What are you talking about? Go code generation is getting better all the time.
  - tedsuo 11y ago
    The translation of the compiler from C to Go was a mostly mechanical transformation, specifically to prevent any changes in the resulting compiled programs. Go compilation is getting better all the time, but the switch itself did not change the resulting programs.
    - kbenson 11y ago
      An article from 2013 means that it covers a change from at least Go version 1.2 (released 2013/12/01)) to 1.5 (released yesterday?). The performance section of each document talks about cases where it may go slower for faster in certain instance in every since release, including the one that was the change from C to Go for the compiler[2]. That said, I took the original statement about Go switching from Co to Go to indicate there's been major changes in the compiler, so it would be interesting to see more recent results, not that the change itself necessarily was responsible for major speed improvements, but they could very well have assumed the compiler change would have had a larger affect on the resulting binary depending on their understanding of the Go toolchain. 1: https://golang.org/doc/devel/release.html https://golang.org/doc/devel/release.html 2: https://golang.org/doc/go1.4#performance https://golang.org/doc/go1.4#performance
- curun1r 11y ago
  It's still hard to take any benchmark seriously when given the quote: > Mosquitto was compiled with gcc 4.8.2, the Go implementation was executed with go run, the D implementation was compiled with dmd 2.0.64.2 and the Erlang version I’m not sure. The "I'm not sure" speaks for itself, but go run also includes both compilation time and execution time and they're comparing it against just the execution times of the other languages. That's not exactly an apples to apples comparison.
humanrebar 11y ago
That was basically my reaction as well. I would also like to see a Rust implementation included.

by nanny 11y ago

I really can't imagine that this benchmark is relevant anymore. Haven't Go, D, and Erlang had major changes since then? E.g., Go is now compiled by a compiler written in Go.

atilaneves 11y ago

I'm planning on updating it this year once I finally find the time to learn Rust. Developing an MQTT broker has become my go-to task for learning new languages.

acqq 11y ago
I'm really glad that you've done the D implementation. I think D gets less attention than it should, especially compared to Go. Maybe somebody already proficient in Rust, reading this, can contribute?
nanny 11y ago
No big deal, it wasn't a slight against you, but rather the OP. It's not your fault he reposted your two year old benchmarks.
pron 11y ago
How about Java, then? It's a little less esoteric than most of those languages.
- atilaneves 11y ago
  Funny you should ask: https://atilanevesoncode.wordpress.com/2014/01/08/adding-java-and-c-to-the-mqtt-benchmarks-or-how-i-learned-to-stop-worrying-and-love-the-garbage-collector/ https://atilanevesoncode.wordpress.com/2014/01/08/adding-jav...
  - pron 11y ago
    So Java is the pingtest winner, and I'm sure that with just a little more work it can be the loadtest winner as well.
    - atilaneves 11y ago
      I doubt it, Patrick properly profiled and optimised it. Later on I got the D version to within 3% of Java's performance in pingtest.
      - pron 11y ago
        It's clearly optimized for latency over throughput. For one, it is single threaded -- unlike the Erlang, Go and, I believe D (vibe.d) implementations. Also, I don't know exactly how the benchmark was performed, but Java takes time to warm up (otherwise you're testing a completely different implementation of Java, which is only used at startup time). I've found that for long-running server-side apps, Java is very hard to beat in terms of performance, especially relative to the effort spent. I also think it's a nicer language than Go (I also prefer it over C++ and D).

rubiquity 11y ago

How Go is compiled isn't as relevant to its performance as much as what Go is compiled into, which hasn't changed all that much as far as I know. The GC improvements might help with latency though.

voidlogic 11y ago
>as much as what Go is compiled into, which hasn't changed all that much as far as I know What are you talking about? Go code generation is getting better all the time.
- tedsuo 11y ago
  The translation of the compiler from C to Go was a mostly mechanical transformation, specifically to prevent any changes in the resulting compiled programs. Go compilation is getting better all the time, but the switch itself did not change the resulting programs.
  - kbenson 11y ago
    An article from 2013 means that it covers a change from at least Go version 1.2 (released 2013/12/01)) to 1.5 (released yesterday?). The performance section of each document talks about cases where it may go slower for faster in certain instance in every since release, including the one that was the change from C to Go for the compiler[2]. That said, I took the original statement about Go switching from Co to Go to indicate there's been major changes in the compiler, so it would be interesting to see more recent results, not that the change itself necessarily was responsible for major speed improvements, but they could very well have assumed the compiler change would have had a larger affect on the resulting binary depending on their understanding of the Go toolchain. 1: https://golang.org/doc/devel/release.html https://golang.org/doc/devel/release.html 2: https://golang.org/doc/go1.4#performance https://golang.org/doc/go1.4#performance
curun1r 11y ago
It's still hard to take any benchmark seriously when given the quote: > Mosquitto was compiled with gcc 4.8.2, the Go implementation was executed with go run, the D implementation was compiled with dmd 2.0.64.2 and the Erlang version I’m not sure. The "I'm not sure" speaks for itself, but go run also includes both compilation time and execution time and they're comparing it against just the execution times of the other languages. That's not exactly an apples to apples comparison.

humanrebar 11y ago

That was basically my reaction as well. I would also like to see a Rust implementation included.