3 ms·

Another day, another "proof" that Ruby is slower than X, which surprises exactly no one. The interesting thing here though is that considering the trivial amou

by relix 13y ago

Another day, another "proof" that Ruby is slower than X, which surprises exactly no one. The interesting thing here though is that considering the trivial amount of work to be done, Ruby shouldn't be that slow. I'd like to see the source code of the Ruby project to see if some simple improvements could have been made that aren't too invasive. The spike in latency at around 15,000 requests could be explained by the garbage collector, which normally is put out-of-band. This makes me believe that he used a single-threaded, single-worker approach for Ruby. I don't know about the Go framework he used, but maybe it was multi-threaded (and not green-threads).

Freaky 13y ago
Puma (http://puma.io/ http://puma.io/) is multithreaded, but only JRuby and Rubinius support concurrent execution - standard MRI/YARV have a global VM lock just like Python.
cheald 13y ago
I kinda sorta suspect he was using ruby-json, which is substantially slower at JSON serialization than something like Oj. It's not going to make the Rails implementation start seriously competing with Go, but it would help matters substantially. require 'benchmark' require 'json' require 'oj' require 'open-uri' TIMES = 100000 hash = JSON.parse(open("https://gist.github.com/gnip/764239/raw/6c6a2297f3e4e29a626f07db0c57b45af7d7e5d7/Twitter+%28json+format%29.js").read) Benchmark.bm do |x| x.report("Oj") { TIMES.times { Oj.dump(hash) } } x.report("ruby-json") { TIMES.times { JSON.dump(hash) } } end # user system total real # Oj 0.980000 0.000000 0.980000 ( 0.983426) # ruby-json 8.780000 0.000000 8.780000 ( 8.832761) Edit: I'm working on a Go implementation of the above benchmark. I haven't touched Go before today, though, so it's a bit of learning. :) Edit edit: Here's the benchmark in Go. First time I've touched it, tell me if I stuffed something up awfully. Amusingly, it ended up being almost exactly 10x many lines as the Ruby code. :) https://gist.github.com/cheald/5784471 https://gist.github.com/cheald/5784471 BenchmarkJsonEncode 50000 53661 ns/op So the final results: Implementation ns/op ------------------------ Ruby (ruby-json) 88,327 Ruby (via Oj) 9,834 Go 53,661 I'm not quite sure that feels right, so if someone else could sanity check me, please do.
bhauer 13y ago
I can't speak to Matt's source code, but you can find ours at GitHub [1]. In fact, the ETA for Round 6 is next Thursday, June 20. If you can point out any tuning for us to put in place for the Ruby tests before June 17, we may be able to get pull requests accepted before we kick off the test runs. [1] https://github.com/TechEmpower/FrameworkBenchmarks https://github.com/TechEmpower/FrameworkBenchmarks
danmaz74 13y ago
The Ruby code is running the full Rails stack. Instead with Go he went almost bare-metal. That alone makes a BIG difference.

by relix 13y ago

Freaky 13y ago

Puma (http://puma.io/ http://puma.io/) is multithreaded, but only JRuby and Rubinius support concurrent execution - standard MRI/YARV have a global VM lock just like Python.

cheald 13y ago

I kinda sorta suspect he was using ruby-json, which is substantially slower at JSON serialization than something like Oj. It's not going to make the Rails implementation start seriously competing with Go, but it would help matters substantially. require 'benchmark' require 'json' require 'oj' require 'open-uri' TIMES = 100000 hash = JSON.parse(open("https://gist.github.com/gnip/764239/raw/6c6a2297f3e4e29a626f07db0c57b45af7d7e5d7/Twitter+%28json+format%29.js").read) Benchmark.bm do |x| x.report("Oj") { TIMES.times { Oj.dump(hash) } } x.report("ruby-json") { TIMES.times { JSON.dump(hash) } } end # user system total real # Oj 0.980000 0.000000 0.980000 ( 0.983426) # ruby-json 8.780000 0.000000 8.780000 ( 8.832761) Edit: I'm working on a Go implementation of the above benchmark. I haven't touched Go before today, though, so it's a bit of learning. :) Edit edit: Here's the benchmark in Go. First time I've touched it, tell me if I stuffed something up awfully. Amusingly, it ended up being almost exactly 10x many lines as the Ruby code. :) https://gist.github.com/cheald/5784471 https://gist.github.com/cheald/5784471 BenchmarkJsonEncode 50000 53661 ns/op So the final results: Implementation ns/op ------------------------ Ruby (ruby-json) 88,327 Ruby (via Oj) 9,834 Go 53,661 I'm not quite sure that feels right, so if someone else could sanity check me, please do.

bhauer 13y ago

I can't speak to Matt's source code, but you can find ours at GitHub [1]. In fact, the ETA for Round 6 is next Thursday, June 20. If you can point out any tuning for us to put in place for the Ruby tests before June 17, we may be able to get pull requests accepted before we kick off the test runs. [1] https://github.com/TechEmpower/FrameworkBenchmarks https://github.com/TechEmpower/FrameworkBenchmarks

danmaz74 13y ago

The Ruby code is running the full Rails stack. Instead with Go he went almost bare-metal. That alone makes a BIG difference.