7 ms·
Benchmarking D vs Go vs Erlang vs C for MQTT broker
- deleted 13y ago[deleted]
- colanderman 13y ago"the Erlang version I’m not sure" At least try to find out if it's HiPE (native compilation) or not?
- lpgauth 13y ago"an excuse to learn Erlang, and wrote his own implementation" Why even included Erlang if you're not going to share the source...
- atilaneves 13y agoHe wasn't in the office. I'm going to ask him about it today. Like it says at the end of the blog post, if he puts it up on bitbucket I'll include the link.
- atilaneves 13y agoThere's a link to it now.
- rdtsc 13y ago> Go 1.2 produces a binary that performs on the order of 10%-15% faster than the numbers above, which might mean equivalent performance to the Erlang implementation. Huh, impressive for Erlang, beating a newer, statically compiled language, while being interpreted and with dynamic types.
- pgeorgi 13y agonewer, statically compiled language with a minimalist, unoptimized compiler versus an interpreter that got decades to mature (and HiPE to speed things up some more)
- pvalsecc 13y agoI didn't use HIPE. It was providing no performance improvement for this use case.
- wcummings 13y agoErlang's VM is very sophisticated and mature, it is at least years ahead of Go, imo. For example, compare Erlang's pre-emptive scheduling [1] to "goroutines". [1] http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-scheduling.html http://jlouisramblings.blogspot.com/2013/01/how-erlang-does-...
- chimeracoder 13y ago> it is at least years ahead of Go, Yes, Erlang has admittedly had decades to improve the scheduler (versus a couple of years for the gc compiler in Go), so this is quite literally true. However, you should know that goroutines are pre-emptively scheduled now; the "scare quotes in your comment seem to imply that they aren't: http://golang.org/doc/go1.2#preemption http://golang.org/doc/go1.2#preemption
- shanemhansen 13y agoI'm a go fanboi, but to be fair, goroutines are now more preemptive, but not completely. In practice that means that if you call a function somewhere in your infinite loop, you'll get preempted. It's still possible for a single goroutine to steal an os thread. Practically speaking though, I feel like nobody should be scared away from go due to it's lack of full pre-emption. Seriously, don't write an infinite loop with no function calls.
- deleted 13y ago[deleted]
- mkhattab 13y agoPerhaps even more impressive is that it was written by an Erlang novice. This makes me want to hit the books and start learning Erlang again.
- atilaneves 13y agoHe's an Erlang novice, but the guy is the best programmer I've met. I have no doubts that if he chose to pick up D instead he'd've written code as fast as mine.
- eerr 13y agoWould have been cool to see how many lines of code each implementation took including the erlang one.
- atilaneves 13y agoIt's mentioned. It's not easy to measure but they're all in the 800-1000 lines region.
- static_typed 13y agoThe key takeaway in the benchmarks seem is that Go is significantly slower than the rest, much much slower. Apart from the regular 30-sec interval postings on HN extolling the virtues of Go, it seems to me, people should rekindle an interest, or discover a new interest in D, C or Erlang where performance is a consideration, and maybe Go where a need to feel like part of the post-Ruby crowd.
- CoffeeDregs 13y ago>Go is significantly slower than the rest, much much slower. Everyone knows that Go is a new-ish language and has room to improve on performance. That said, Go seems perfectly fast since all of the languages seem to perform fairly well. Further, were Go "much much slower", CloudFlare, IronIO, Google, etc would probably not be using it... >discover a new interest in D, C or Erlang >where performance is a consideration Erlang on The Game: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=all&lang=go&lang2=hipe&data=u64q http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... Go looks to be about ~50% the performance of C++, but, from lots of peoples' comments/writings, is much nicer to write so they use it when productivity is more important than absolute performance. Note: I'm interested in Go (as I am about lots of languages), but am not a user of it right now.
- colanderman 13y agoYa, Erlang doesn't do well on the Benchmarks Game because the Benchmarks Game doesn't measure communication, which is Erlang's strong point. No-one writes algorithms like those in the Game in Erlang; you code them as a server in C or Java or anything and farm work out to them from Erlang.
- rdtsc 13y ago> Erlang on The Game: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t.. http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t.... The Game is a terrible benchmark for Erlang. Math, computing digits if Pi, string searching, n-body problems. If that is the target domain, don't pick Erlang, or Go, pick Fortran or C.
- 13y ago
- drakaal 13y agoSummary: I hate Go, So Go Lost this race. I love D, So it won or tied with C which I don't like but everyone knows is fast. I wanted to learn Erlang, so I made sure it did better than Go. I didn't share my SourceCode so you can't see if I was stupid or cheated. Did I miss anything?
- EugeneOZ 13y agoAlso there are bugs in my Go-version test, I know about them but I hate Go so, pfff, who cares.
- stinkypete 13y agoSource links for everything but the Erlang version are there at the end of the article.
- andralex 13y agoQuite a few things. The Go version was written by a person enthusiastic and presumably competent in Go. See their perspective here where he concedes defeat on the performance front: http://blog.nella.org/mqtt-code-golf/ http://blog.nella.org/mqtt-code-golf/ Adding C as a baseline is entirely appropriate. The source code for C, Go, and D is available. The Erlang version may or may not at the discretion of its author. Probably a more sensible hypothesis is that you like Go and saw a result that is unfavorable to it, so you had an emotional response.
- azth 13y ago
- codygman 13y ago> Go implementation was executed with go run Why didn't they just use "go build program.go; ./program"? If you're compiling the other programs it doesn't seem fair to use the interpreter for go.
- colanderman 13y agoI thought "go run" did exactly that (compile then run)?
- codygman 13y agoPerhaps I'm mistaken, looking at the source it does seem to do just that: http://golang.org/src/cmd/go/run.go?ModPagespeed=noscript http://golang.org/src/cmd/go/run.go?ModPagespeed=noscript
- f2f 13y agoi see no mention of "concurrency" in that article. i see that the go programs make heavy use of goroutines so why were they run in "single threaded" mode? just testing their mqttsrv with various levels of GOMAXPROCS rewards with immediate benefits. here are some numbers for pingtest and loadtest in Go for varying levels of GOMAXPROCS for them and the server: (pingtest/mqtt gomaxprocs) vs pingtest, reporting "elapsed time": 1x1 5.8s 2x1 6.0s 4x1 6.0s 8x1 6.1s 16x1 6.1s 1x2 5.5s 2x2 3.5s 4x2 3.4s 8x2 3.5s 16x2 3.5s 1x4 5.4s 2x4 3.6s 4x4 3.3s 8x4 3.4s 16x4 3.7s same thing but for loadtest, reported value is "messages/sec": 1x1 75521 2x1 76523 4x1 77477 8x1 74613 16x1 73678 1x2 71439 2x2 100571 4x2 93426 8x2 108938 16x2 97593 1x4 71390 2x4 87426 4x4 98746 8x4 99320 16x4 119692 1x8 64413 2x8 87445 4x8 102953 8x8 120720 16x8 106842 (note that the loadtest doesn't look like it takes long enough to be meaningful in its default) as always with benchmarks, caveat emptor, but in my opinion go does just fine.
- yelnatz 13y agoAre your numbers in line with the blog's table? So you get those instead of these? Loadtest: Go: 90.9 +/- 11 100.1 +/- 0.1 99.3 +/- 0.2 98.8 +/- 0.3 If not can you make it relevant to the table so people can compare and contrast?
- f2f 13y agoloadtest for 100 500 750 1000, X*Y below means X = loadtest GOMAXPROCS and Y = mqttsrv GOMAXPROCS: 1x1 80735 67696 67068 66585 2x1 64380 76531 31346 43268 4x1 77613 73794 72759 43263 8x1 77813 64197 32899 46960 1x2 71641 72521 72715 72695 2x2 103711 102944 104210 39619 4x2 73171 23803 90537 46204 8x2 96624 49698 33714 45729 1x4 69418 74088 34364 38935 2x4 88805 102847 113740 108829 4x4 102406 108110 34672 120161 8x4 109022 124726 121642 46765 1x8 62336 68238 71297 68754 2x8 104833 115276 118170 44617 4x8 122292 126075 126101 121306 8x8 108826 125098 124765 47713 edit: s/pingtest/loadtest/ in the explanation above.
- yelnatz 13y ago
- ader277 13y ago"the Go implementation was executed with go run ". I think it would be much more appropriate to build the GO script with "go build" and use the resultant binary. Because the D program was compiled.
- colanderman 13y agohttp://golang.org/src/cmd/go/run.go?ModPagespeed=noscript http://golang.org/src/cmd/go/run.go?ModPagespeed=noscript "Run compiles and runs the main package comprising the named Go source files."
- deleted 13y ago[deleted]
- he_the_great 13y agoSo he should have run the D program with dmd -run :D
- jeremiep 13y agoor #!/usr/bin/rdmd
- Thaxll 13y agoI don't understand those benchmarks, 'insert any languages' can achieve performances than 99% of companies don't even need. And for the other 1% you're probably smarter than those posts on HN.
- karcass 13y ago"Go is an opinionated language, which would be fine if I agreed with its creators’ opinions." Spot on.
- chrismorgan 13y agoI found this article a little hard to read; - #555 on #fff is distinctly too light for a web page's text color (I generally recommend around #202020); - the font-size of 13px is too small—remove it and let the default of 16px show.
- chrismorgan 13y agoI'd rather like to see Rust in here.
- atilaneves 13y agoMe too, but I've decided to not even look at Rust until it hits 1.0.
- kibwen 13y agoGood, because we're certainly not ready to compete in serious benchmarks yet. :)
- emrekzd 13y agoWhile your "benchmark" (if we should call it that way) obviously make no sense, I would suggest you guys look into how concurrency and parallelism are different concepts.
- dscrd 13y agoThe results of the latency benchmark say that a larger latency is better. Is that a mistake?
- atilaneves 13y agoNo, it was measured in thousands of messages per second.