4 ms·

Until recently none of the Ruby setups in the TechEmpower benchmark were configured even vaguely correctly. Previously they were comparing Sinatra running on 8

Until recently none of the Ruby setups in the TechEmpower benchmark were configured even vaguely correctly. Previously they were comparing Sinatra running on 8 cores to Go running on 40 cores. At my day job have an a prod analytics service with all the stuff you mention written in Rust + PG, and the Rust bit which handles data ingestion via HTTP and JSON is only about 5x faster than the equivalent Ruby code. Obviously, that was a big disappoitment to me. Serialization performance in the default Rails setup is horrific but calling Oj explicitly on an ActiveModelSerializer instead of the Rails "render :json, @model" brings it much closer to "fast" languages than you'd expect, improving performance about 74x. I wish I was joking. If the perf difference simply means buying 2x the servers then as a company you have to be pretty big before it's worth throwing out the entire Ruby/Rails ecosystem and getting stuck with far less mature libraries in newer, less widely used languages like Go or Rust. There are tons of major sites using Ruby/Python for their web layer. Instagram, Stripe, Airbnb, Shopify, HotelTonight, GitHub, and even part of Netflix is powered by Rails. If you look at the work Aaron Patterson and Shaun Griffin have done and are doing on AR, or what Peter Ohler has done with Oj, I don't think it's fair to say that either the Rails or Ruby ecosystem doesn't care about performance. They just don't put it before usability by default. Compared to other scripting languages, even performance focused ones like Lua, Ruby's performance is very good, and will only continue to improve with MJIT coming to MRI Ruby, and an entire new high performance alternative implementation coming with TruffleRuby.

ehsanu1 9y ago
At my day job have an a prod analytics service with all the stuff you mention written in Rust + PG, and the Rust bit which handles data ingestion via HTTP and JSON is only about 5x faster than the equivalent Ruby code. Obviously, that was a big disappoitment to me. 5x isn't too shabby for something potentially somewhat IO-bound. Presumably the effort wasn't 5x to build and maintain that, so if that's a very hot inner loop of something that's great. What I've found is that using Ruby (mainly gems) idiomatically can just lead to way slower performance. Eg I use time libraries a lot for my day job, and the idiomatic way to write the code had tons of hidden costs and was basically impossible to optimize. So I wrote a ruby extension in rust, and got a 30x speedup on that code - it was pretty computationally intensive, so a perfect fit for this. Part of it was that I was forced to use more primitive types, but in Rust-land I was able to add types all over the primitives and abstract away to my heart's content, without any overhead. That's just something that's impossible in Ruby - each abstraction always has a cost and there's no way around it. If you look at the work Aaron Patterson and Shaun Griffin have done and are doing on AR, or what Peter Ohler has done with Oj, I don't think it's fair to say that either the Rails or Ruby ecosystem doesn't care about performance. They just don't put it before usability by default. Right I wouldn't say nobody cares about performance, some people care a lot and they've been doing great work and helping raise the bar. But overall, the bar is pretty low in the Ruby world. Chris Seaton (one of TruffleRuby's main authors) has a talk where he shows some of the batshit insane code you see in gems, which they had to figure out how to optimize in TruffleRuby/Graal. Code like allocating a new array and calling the `min` method in order to find the smaller of two values! And that was being called in a hot inner loop! In a gem with otherwise great functionality, a nice website and thousands of users. That's the kind of thing you only see in Ruby-land. Also, using method_missing (gotta have that pretty DSL, performance and greppability/discoverability be damned) and monkey-patching/alias-chaining/etc is still common unfortunately (though a smart JIT, ie Graal, can apparently handle that). I am really excited about TruffleRuby and it does really amazing things to optimize crazy Ruby code, so I'm looking forward to Ruby being as fast as say JS in the future. Can't wait for the day I can turn on TruffleRuby in production and halve the AWS bill.
- jashmatthews 9y ago
  Both are CPU bound on de-serialization. It’s basically DB replication over HTTP plus some business logic at either end. The Sinatra version is built in a weekend which is 5x slower has authentication and does use ActiveRecord. Building an entire web app, with background job processing, in Rust with the ecosystem how it is at the moment is very, very slow compared to Ruby, Sinatra and Sidekiq. When we started Diesel couldn’t join to more than one table. Probably approaching 5x in development time. It’s just not mature enough yet. We’re still experiencing stuck threads which don’t timeout. Sounds like you found a great case for a native extension!
  - ehsanu1 9y ago
    Agreed, I wouldn't do a production web service purely in Rust yet, the ecosystem is still way too immature.

ehsanu1 9y ago

At my day job have an a prod analytics service with all the stuff you mention written in Rust + PG, and the Rust bit which handles data ingestion via HTTP and JSON is only about 5x faster than the equivalent Ruby code. Obviously, that was a big disappoitment to me. 5x isn't too shabby for something potentially somewhat IO-bound. Presumably the effort wasn't 5x to build and maintain that, so if that's a very hot inner loop of something that's great. What I've found is that using Ruby (mainly gems) idiomatically can just lead to way slower performance. Eg I use time libraries a lot for my day job, and the idiomatic way to write the code had tons of hidden costs and was basically impossible to optimize. So I wrote a ruby extension in rust, and got a 30x speedup on that code - it was pretty computationally intensive, so a perfect fit for this. Part of it was that I was forced to use more primitive types, but in Rust-land I was able to add types all over the primitives and abstract away to my heart's content, without any overhead. That's just something that's impossible in Ruby - each abstraction always has a cost and there's no way around it. If you look at the work Aaron Patterson and Shaun Griffin have done and are doing on AR, or what Peter Ohler has done with Oj, I don't think it's fair to say that either the Rails or Ruby ecosystem doesn't care about performance. They just don't put it before usability by default. Right I wouldn't say nobody cares about performance, some people care a lot and they've been doing great work and helping raise the bar. But overall, the bar is pretty low in the Ruby world. Chris Seaton (one of TruffleRuby's main authors) has a talk where he shows some of the batshit insane code you see in gems, which they had to figure out how to optimize in TruffleRuby/Graal. Code like allocating a new array and calling the `min` method in order to find the smaller of two values! And that was being called in a hot inner loop! In a gem with otherwise great functionality, a nice website and thousands of users. That's the kind of thing you only see in Ruby-land. Also, using method_missing (gotta have that pretty DSL, performance and greppability/discoverability be damned) and monkey-patching/alias-chaining/etc is still common unfortunately (though a smart JIT, ie Graal, can apparently handle that). I am really excited about TruffleRuby and it does really amazing things to optimize crazy Ruby code, so I'm looking forward to Ruby being as fast as say JS in the future. Can't wait for the day I can turn on TruffleRuby in production and halve the AWS bill.

jashmatthews 9y ago
Both are CPU bound on de-serialization. It’s basically DB replication over HTTP plus some business logic at either end. The Sinatra version is built in a weekend which is 5x slower has authentication and does use ActiveRecord. Building an entire web app, with background job processing, in Rust with the ecosystem how it is at the moment is very, very slow compared to Ruby, Sinatra and Sidekiq. When we started Diesel couldn’t join to more than one table. Probably approaching 5x in development time. It’s just not mature enough yet. We’re still experiencing stuck threads which don’t timeout. Sounds like you found a great case for a native extension!
- ehsanu1 9y ago
  Agreed, I wouldn't do a production web service purely in Rust yet, the ecosystem is still way too immature.