7 ms·

You can't always add a few more servers. For example, let's say you have a per-request processing budget of 100ms. The efficiency of the generated code for you

You can't always add a few more servers. For example, let's say you have a per-request processing budget of 100ms. The efficiency of the generated code for your language of choice effectively limits the amount of (non-concurrent, single threaded) code you can run while processing said request. It's exactly this limit that eventually leads engineering teams at your Dropboxes, Facebooks, Twitters, Tumblrs, etc. to start investigating alternatives or alternative implementations of their favorite dynamic languages (PHP/Python/Ruby -> HHVM/Pyston/?).

NoPiece 9y ago
It's exactly this limit that eventually leads engineering teams at your Dropboxes, Facebooks, Twitters, Tumblrs True, but those are such extreme edge cases. 99.9999% of projects will never see that kind of demand for scale and speed. Facebook shouldn't switch to Ruby, but that doesn't inform whether Ruby would be viable for many (most) projects.
- nirvdrum 9y ago
  I don't think it's that much of an edge case. If Rails and its ecosystem lead to 300ms render times without load, but your objective is 100ms, you're going to have a problem. I don't think either number here is extreme and I've encountered that sort of situation on previous projects. It's not easily solvable by throwing more machines at it, unfortunately.
  - tomc1985 9y ago
    If Rails and your ecosystem are pushing 300ms renders you have bigger problems on your plate
    - nirvdrum 9y ago
      I don't think it's uncommon. But the point is it usually has little to do with load and isn't easily solved by scaling horizontally. Sure, you might be able to gain some efficiencies with distributed caches and such, but that still requires a fair bit of effort (the developer time we're trying to minimize). You're right though. And I managed to sidetrack myself a bit. My intention in participating in the thread was to point out that you can't always optimize for developer speed and think you can get out of it "cheaply" with more hardware. And it's not just a problem when you're big enough for it to be a good problem to have. In those cases you're going to need to find a way to make the application faster. Maybe it's aggressive in-app caching; maybe it's sitting there analyzing application profiles; maybe it's a rewrite in another language or another framework. Our goal is to provide a fast enough runtime where you don't have to make that decision.
      - tomc1985 9y ago
        > the point is it usually has little to do with load and isn't easily solved by scaling horizontally. My point exactly. Unless you're getting slammed with traffic, there's too much going on in a page load if it takes 300ms to spit out.
        nirvdrum 9y ago
        If you can achieve the same rendered output much faster with a different runtime or with a rewrite or with a different framework, I don't think you're looking at a fundamental design flaw with the page. I'm happy to agree to disagree. Maybe there is too much going on with the page. But it's a common problem with Rails apps and is ideally solved without a rewrite. I think a faster runtime would be the ideal solution here.
    - kuschku 9y ago
      At my day job we've seen load times for pages > 5 seconds in rails. There's no way to do it faster because the data has to be dynamically generated and loaded, and that's the bottleneck. It's not uncommon to see this in rails applications — GitLab had the same issues.
      - jashmatthews 9y ago
        In GitLab's case, a lot of their issues are just the classic self inflicted N+1 queries and poor planning of how to represent events in a way that could be quickly queried for display. OTOH, the background job memory issues were completely not their fault. Glibc malloc trades high memory usage for keeping both high throughput and simplicity. With Sidekiq or puma this causes much higher memory usage than necessary. Switching to jemalloc, which they did, can reduce memory usage by 50-75%. What does your app do that takes 5 seconds? Is this because hundreds of DB calls are required, or you need to serialize thousands of objects?
  - jashmatthews 9y ago
    Out of pure curiosity what took 300ms? Image composition? Huge JSON payload serialization?
    - ehsanu1 9y ago
      I've seen numerous big rails apps with times as long or longer, even after decent effort has been put in to optimize. Generally a case of having a lot of things on a page and rendering it all into html server-side (the classic/old rails way of doing things).
      - jashmatthews 9y ago
        300ms from server side rendering requires something self inflicted. Even bulk JSON endpoints should be faster than that. 27ms is the average dynamic response time for Basecamp, which is server side rendered HTML done the classic way. I worked on a high performance Rails API serving dynamically generated map tiles and our response times were actually lower than that.
        nirvdrum 9y ago
        Doesn't basecamp do a ton of caching to achieve response times like that? There are certainly techniques to get there, but how long does it take to populate the caches? That's what you're going to see out of a stock Rails app. Self-inflicted, sure. But in my experience it's the result of picking developer-friendly tools to optimize for developer time. Erubis is much faster than HAML, but a lot of developers prefer the latter. Those sorts of decisions accumulate.
        jashmatthews 9y ago
        No more than Match.com do, who run on .NET. Basecamp just uses the standard Rails caching AFAIK. Templating just isn't the issue now that it was in the days of 1.8. HAML is slower than ERB, but usually neither have a meaningful impact on response time on modern Ruby. ERB is approaching Erubis performance now.
jashmatthews 9y ago
I work for a company exploring Rust and Go at the moment and it turns out there's little to be gained in moving away from Ruby for web app performance. Go + Gin is only about 2x faster than a modern Ruby + Sinatra + at "serialize some stuff to JSON and respond over HTTP" and JRuby closes the gap even further https://www.techempower.com/benchmarks/previews/round15/#section=data-r15&hw=ph&test=json&l=zijl6n https://www.techempower.com/benchmarks/previews/round15/#sec... There are cases where Ruby is obviously very slow compared to Go or an AOT langauge like Rust but serving dynamic web requests isn't really one of them. This wasn't true when Twitter moved away from Ruby 1.8 but a huge amount of work has gone into Ruby since then. For example: totally new VM, GC, threading system and there's currently an open PR to add a basic JIT compiler.
- ehsanu1 9y ago
  Those Sinatra numbers look a lot better than I remember them being! But I wouldn't give that much credence to that, there's barely anything happening in those benchmarks and it seems to be testing things that have had a lot of optimization effort thrown at them. Try adding session management, user permissions, large data models, some business logic, all without putting a lot of effort into optimizing (ie like most work environments) and try again. Also, yeah, use some heavy (extremely popular) gems like ActiveRecord. And God forbid you actually do something computationally intensive anywhere. Serialization (even with drivers written in C) is also very slow if you have a lot of data. And if you have enough traffic, even a 2x performance difference can be pretty important in terms of cost. AR and similar libraries are the killer of performance in the Ruby eco-system. It's not the bare web framework itself, but the entire eco-system is generally not too worried about performance compared to other languages (partially because it's harder to optimize in Ruby, partially due to the culture). That said, the language has gotten a lot faster since the 1.8 days, eg GC has seen a lot of improvement, but it's still damn slow.
  - jashmatthews 9y ago
    Until recently none of the Ruby setups in the TechEmpower benchmark were configured even vaguely correctly. Previously they were comparing Sinatra running on 8 cores to Go running on 40 cores. At my day job have an a prod analytics service with all the stuff you mention written in Rust + PG, and the Rust bit which handles data ingestion via HTTP and JSON is only about 5x faster than the equivalent Ruby code. Obviously, that was a big disappoitment to me. Serialization performance in the default Rails setup is horrific but calling Oj explicitly on an ActiveModelSerializer instead of the Rails "render :json, @model" brings it much closer to "fast" languages than you'd expect, improving performance about 74x. I wish I was joking. If the perf difference simply means buying 2x the servers then as a company you have to be pretty big before it's worth throwing out the entire Ruby/Rails ecosystem and getting stuck with far less mature libraries in newer, less widely used languages like Go or Rust. There are tons of major sites using Ruby/Python for their web layer. Instagram, Stripe, Airbnb, Shopify, HotelTonight, GitHub, and even part of Netflix is powered by Rails. If you look at the work Aaron Patterson and Shaun Griffin have done and are doing on AR, or what Peter Ohler has done with Oj, I don't think it's fair to say that either the Rails or Ruby ecosystem doesn't care about performance. They just don't put it before usability by default. Compared to other scripting languages, even performance focused ones like Lua, Ruby's performance is very good, and will only continue to improve with MJIT coming to MRI Ruby, and an entire new high performance alternative implementation coming with TruffleRuby.

NoPiece 9y ago

It's exactly this limit that eventually leads engineering teams at your Dropboxes, Facebooks, Twitters, Tumblrs True, but those are such extreme edge cases. 99.9999% of projects will never see that kind of demand for scale and speed. Facebook shouldn't switch to Ruby, but that doesn't inform whether Ruby would be viable for many (most) projects.

nirvdrum 9y ago
I don't think it's that much of an edge case. If Rails and its ecosystem lead to 300ms render times without load, but your objective is 100ms, you're going to have a problem. I don't think either number here is extreme and I've encountered that sort of situation on previous projects. It's not easily solvable by throwing more machines at it, unfortunately.
- tomc1985 9y ago
  If Rails and your ecosystem are pushing 300ms renders you have bigger problems on your plate
  - nirvdrum 9y ago
    I don't think it's uncommon. But the point is it usually has little to do with load and isn't easily solved by scaling horizontally. Sure, you might be able to gain some efficiencies with distributed caches and such, but that still requires a fair bit of effort (the developer time we're trying to minimize). You're right though. And I managed to sidetrack myself a bit. My intention in participating in the thread was to point out that you can't always optimize for developer speed and think you can get out of it "cheaply" with more hardware. And it's not just a problem when you're big enough for it to be a good problem to have. In those cases you're going to need to find a way to make the application faster. Maybe it's aggressive in-app caching; maybe it's sitting there analyzing application profiles; maybe it's a rewrite in another language or another framework. Our goal is to provide a fast enough runtime where you don't have to make that decision.
    - tomc1985 9y ago
      > the point is it usually has little to do with load and isn't easily solved by scaling horizontally. My point exactly. Unless you're getting slammed with traffic, there's too much going on in a page load if it takes 300ms to spit out.
      - nirvdrum 9y ago
        If you can achieve the same rendered output much faster with a different runtime or with a rewrite or with a different framework, I don't think you're looking at a fundamental design flaw with the page. I'm happy to agree to disagree. Maybe there is too much going on with the page. But it's a common problem with Rails apps and is ideally solved without a rewrite. I think a faster runtime would be the ideal solution here.
  - kuschku 9y ago
    At my day job we've seen load times for pages > 5 seconds in rails. There's no way to do it faster because the data has to be dynamically generated and loaded, and that's the bottleneck. It's not uncommon to see this in rails applications — GitLab had the same issues.
    - jashmatthews 9y ago
      In GitLab's case, a lot of their issues are just the classic self inflicted N+1 queries and poor planning of how to represent events in a way that could be quickly queried for display. OTOH, the background job memory issues were completely not their fault. Glibc malloc trades high memory usage for keeping both high throughput and simplicity. With Sidekiq or puma this causes much higher memory usage than necessary. Switching to jemalloc, which they did, can reduce memory usage by 50-75%. What does your app do that takes 5 seconds? Is this because hundreds of DB calls are required, or you need to serialize thousands of objects?
- jashmatthews 9y ago
  Out of pure curiosity what took 300ms? Image composition? Huge JSON payload serialization?
  - ehsanu1 9y ago
    I've seen numerous big rails apps with times as long or longer, even after decent effort has been put in to optimize. Generally a case of having a lot of things on a page and rendering it all into html server-side (the classic/old rails way of doing things).
    - jashmatthews 9y ago
      300ms from server side rendering requires something self inflicted. Even bulk JSON endpoints should be faster than that. 27ms is the average dynamic response time for Basecamp, which is server side rendered HTML done the classic way. I worked on a high performance Rails API serving dynamically generated map tiles and our response times were actually lower than that.
      - nirvdrum 9y ago
        Doesn't basecamp do a ton of caching to achieve response times like that? There are certainly techniques to get there, but how long does it take to populate the caches? That's what you're going to see out of a stock Rails app. Self-inflicted, sure. But in my experience it's the result of picking developer-friendly tools to optimize for developer time. Erubis is much faster than HAML, but a lot of developers prefer the latter. Those sorts of decisions accumulate.
        jashmatthews 9y ago
        No more than Match.com do, who run on .NET. Basecamp just uses the standard Rails caching AFAIK. Templating just isn't the issue now that it was in the days of 1.8. HAML is slower than ERB, but usually neither have a meaningful impact on response time on modern Ruby. ERB is approaching Erubis performance now.

jashmatthews 9y ago

I work for a company exploring Rust and Go at the moment and it turns out there's little to be gained in moving away from Ruby for web app performance. Go + Gin is only about 2x faster than a modern Ruby + Sinatra + at "serialize some stuff to JSON and respond over HTTP" and JRuby closes the gap even further https://www.techempower.com/benchmarks/previews/round15/#section=data-r15&hw=ph&test=json&l=zijl6n https://www.techempower.com/benchmarks/previews/round15/#sec... There are cases where Ruby is obviously very slow compared to Go or an AOT langauge like Rust but serving dynamic web requests isn't really one of them. This wasn't true when Twitter moved away from Ruby 1.8 but a huge amount of work has gone into Ruby since then. For example: totally new VM, GC, threading system and there's currently an open PR to add a basic JIT compiler.

ehsanu1 9y ago
Those Sinatra numbers look a lot better than I remember them being! But I wouldn't give that much credence to that, there's barely anything happening in those benchmarks and it seems to be testing things that have had a lot of optimization effort thrown at them. Try adding session management, user permissions, large data models, some business logic, all without putting a lot of effort into optimizing (ie like most work environments) and try again. Also, yeah, use some heavy (extremely popular) gems like ActiveRecord. And God forbid you actually do something computationally intensive anywhere. Serialization (even with drivers written in C) is also very slow if you have a lot of data. And if you have enough traffic, even a 2x performance difference can be pretty important in terms of cost. AR and similar libraries are the killer of performance in the Ruby eco-system. It's not the bare web framework itself, but the entire eco-system is generally not too worried about performance compared to other languages (partially because it's harder to optimize in Ruby, partially due to the culture). That said, the language has gotten a lot faster since the 1.8 days, eg GC has seen a lot of improvement, but it's still damn slow.
- jashmatthews 9y ago
  Until recently none of the Ruby setups in the TechEmpower benchmark were configured even vaguely correctly. Previously they were comparing Sinatra running on 8 cores to Go running on 40 cores. At my day job have an a prod analytics service with all the stuff you mention written in Rust + PG, and the Rust bit which handles data ingestion via HTTP and JSON is only about 5x faster than the equivalent Ruby code. Obviously, that was a big disappoitment to me. Serialization performance in the default Rails setup is horrific but calling Oj explicitly on an ActiveModelSerializer instead of the Rails "render :json, @model" brings it much closer to "fast" languages than you'd expect, improving performance about 74x. I wish I was joking. If the perf difference simply means buying 2x the servers then as a company you have to be pretty big before it's worth throwing out the entire Ruby/Rails ecosystem and getting stuck with far less mature libraries in newer, less widely used languages like Go or Rust. There are tons of major sites using Ruby/Python for their web layer. Instagram, Stripe, Airbnb, Shopify, HotelTonight, GitHub, and even part of Netflix is powered by Rails. If you look at the work Aaron Patterson and Shaun Griffin have done and are doing on AR, or what Peter Ohler has done with Oj, I don't think it's fair to say that either the Rails or Ruby ecosystem doesn't care about performance. They just don't put it before usability by default. Compared to other scripting languages, even performance focused ones like Lua, Ruby's performance is very good, and will only continue to improve with MJIT coming to MRI Ruby, and an entire new high performance alternative implementation coming with TruffleRuby.