3 ms·

Is your Rails app under-provisioned?

35 points by itsderek23 17y ago

minalecs 17y ago
is there anyone currently using scout that can comment on this utility ?
- jharrison 17y ago
  I have been a Scout user for a while now. I primarily use it for its Rails monitoring. They also have a Passenger plugin that monitors a number of Passenger stats. My setup doesn't lend itself to Apache log monitoring (which is what the article refers to). As a monitoring app, I have no complaints but also have little to compare it to. It's worth a look and hard to beat on price, in my opinion. I'm not someone who would roll his own monitoring system so it works well for me.
patio11 17y ago
I signed up for Scout after reading this. It made me realize that this is a plausible soft-failure for my setup and that if it happened I would be totally in the dark until customers started complaining. Granted, I should be so lucky as to have load problems, but if I had nice pretty graphs I could plan future VPS upgrades such that it never becomes a problem for my customers.
- reitzensteinm 17y ago
  How spikey is your traffic? I would have assumed that SEO and Adwords buys would give roughly consistant traffic week after week. I think with that kind of traffic profile you'd almost certainly notice it yourself. Where it would really hurt would be a sudden massive traffic spike, where your traffic rises above the magic number of hits per second that your rails app servers can deal with, and the delay would spiral out of control, only contained by connection timeouts. Before I had properly setup my Django site, I experienced an issue like this with Apache worker threads, and that's exactly what happened. It went undetected until we had 100% higher traffic than normal (at the time, normal was growing by 10% a week), and instead of the site being snappy, most connections were timing out. I was completely out of my depth, having never administered a server before, let alone a dynamic web app with high traffic. Learning as I went while losing 10+ players per second was not fun.
  - patio11 17y ago
    I'm less worried about traffic spikiness and more worried about unanticipated performance implications of changes I make in summer suddenly biting me on the hindquarters come Halloween. (Peak load and also peak sales, and while I think I should have absolutely no problems, I don't have any way to substantiate that and I don't have anything which would tell me I'm incorrect if the server starts suffering performance degradation but all services stay minimally responsive.)
jluxenberg 17y ago
A coworker and I were discussing this yesterday. Our Rails processes take about 50MB of memory each, and can handle only one request. Since our site hits a backend DB that is sometimes slow to return results, requests can last minutes. We could build an out-of-band process to query the backend service and return results asynchronously, but that seems like overkill. However, it's easy for a handful of users to eat up all the connections quickly, if they are all making long-running requests. It seems silly to have an entire Rails process tied up blocked on results from this backend service. And it seems like a rather severe limitation of Rails' architecture to require 50MB of memory per connection-handling process. How do large sites scale Rails?
- jplewicke 17y ago
  I haven't used it myself, but you may want to take a look at Mike Perham's Phat. It modifies Rails to use asynchronous I/O and several cooperating Fibers per-thread/process to speed up Rails significantly. http://www.mikeperham.com/2010/04/03/introducing-phat-an-asynchronous-rails-app/ http://www.mikeperham.com/2010/04/03/introducing-phat-an-asy... http://news.ycombinator.com/item?id=1240609 http://news.ycombinator.com/item?id=1240609 It
- kd5bjo 17y ago
  Scroll down to the web architecture section to see how we do it at Justin.tv: http://highscalability.com/blog/2010/3/16/justintvs-live-video-broadcasting-architecture.html http://highscalability.com/blog/2010/3/16/justintvs-live-vid...

35 points by itsderek23 17y ago

minalecs 17y ago

is there anyone currently using scout that can comment on this utility ?

jharrison 17y ago
I have been a Scout user for a while now. I primarily use it for its Rails monitoring. They also have a Passenger plugin that monitors a number of Passenger stats. My setup doesn't lend itself to Apache log monitoring (which is what the article refers to). As a monitoring app, I have no complaints but also have little to compare it to. It's worth a look and hard to beat on price, in my opinion. I'm not someone who would roll his own monitoring system so it works well for me.

patio11 17y ago

I signed up for Scout after reading this. It made me realize that this is a plausible soft-failure for my setup and that if it happened I would be totally in the dark until customers started complaining. Granted, I should be so lucky as to have load problems, but if I had nice pretty graphs I could plan future VPS upgrades such that it never becomes a problem for my customers.

reitzensteinm 17y ago
How spikey is your traffic? I would have assumed that SEO and Adwords buys would give roughly consistant traffic week after week. I think with that kind of traffic profile you'd almost certainly notice it yourself. Where it would really hurt would be a sudden massive traffic spike, where your traffic rises above the magic number of hits per second that your rails app servers can deal with, and the delay would spiral out of control, only contained by connection timeouts. Before I had properly setup my Django site, I experienced an issue like this with Apache worker threads, and that's exactly what happened. It went undetected until we had 100% higher traffic than normal (at the time, normal was growing by 10% a week), and instead of the site being snappy, most connections were timing out. I was completely out of my depth, having never administered a server before, let alone a dynamic web app with high traffic. Learning as I went while losing 10+ players per second was not fun.
- patio11 17y ago
  I'm less worried about traffic spikiness and more worried about unanticipated performance implications of changes I make in summer suddenly biting me on the hindquarters come Halloween. (Peak load and also peak sales, and while I think I should have absolutely no problems, I don't have any way to substantiate that and I don't have anything which would tell me I'm incorrect if the server starts suffering performance degradation but all services stay minimally responsive.)

jluxenberg 17y ago

A coworker and I were discussing this yesterday. Our Rails processes take about 50MB of memory each, and can handle only one request. Since our site hits a backend DB that is sometimes slow to return results, requests can last minutes. We could build an out-of-band process to query the backend service and return results asynchronously, but that seems like overkill. However, it's easy for a handful of users to eat up all the connections quickly, if they are all making long-running requests. It seems silly to have an entire Rails process tied up blocked on results from this backend service. And it seems like a rather severe limitation of Rails' architecture to require 50MB of memory per connection-handling process. How do large sites scale Rails?

jplewicke 17y ago
I haven't used it myself, but you may want to take a look at Mike Perham's Phat. It modifies Rails to use asynchronous I/O and several cooperating Fibers per-thread/process to speed up Rails significantly. http://www.mikeperham.com/2010/04/03/introducing-phat-an-asynchronous-rails-app/ http://www.mikeperham.com/2010/04/03/introducing-phat-an-asy... http://news.ycombinator.com/item?id=1240609 http://news.ycombinator.com/item?id=1240609 It
kd5bjo 17y ago
Scroll down to the web architecture section to see how we do it at Justin.tv: http://highscalability.com/blog/2010/3/16/justintvs-live-video-broadcasting-architecture.html http://highscalability.com/blog/2010/3/16/justintvs-live-vid...