3 ms·

This extremely simple solution works nicely for us: $load=file_get_contents('/proc/loadavg')[0]; if ($load>10) die('Server load currently too high.');

by TekMol 9y ago

This extremely simple solution works nicely for us: $load=file_get_contents('/proc/loadavg')[0]; if ($load>10) die('Server load currently too high.'); So when the load gets too high, the server will not procss the request. This will bring the server load down. An easy self regulating system.

nkozyra 9y ago
But by this point the request has already been accepted and processed. It's also using a rolling average, which means you'll be "killing" connections that otherwise wouldn't need to be killed.
- TekMol 9y ago
  Well, there is no exact threshold what "needs to bee killed". The higher the load the longer it takes to process requests. At some point you rather want to drop additional requests instead of making things even slower for everybody. This is what happens here. Think of it like this: When the server is already darn slow at a load of 10 we don't want to process more requests because we know at values over 10 it gets unbearable.
  - nkozyra 9y ago
    My point is it's still processing the request. You're just shielding the web server from whatever the application holds. In other words, let's find the theoretical limit to what your web server can handle in terms of concurrent requests. Let's throw them all at your application. In this scenario, the check-load-then-die code is worthless because the request will never get there. It doesn't solve the problem as shown in the OP. What it does is mitigate the effect of the application on the total load of the web server. Which is another problem altogether. Also: > drop additional requests instead of making things even slower for everybody. This is what happens here. Not exactly. The request completes like any HTTP request would. That you're die()ing out doesn't change that, at least from the perspective of the web server.
    - TekMol 9y ago
      find the theoretical limit to what your web server can handle in terms of concurrent requests. Let's throw them all at your application. In this scenario, the check-load-then-die code is worthless because the request will never get there. If there is such a limit, then why not mitigate that by setting a max number of connections for the webserver? You're just shielding the web server from whatever the application holds. True. But as I said: it works. Never seen our server get slow since we implemented this a few years ago. It kicks in when we get about 25x our normal traffic. Which happened less then once a year so far.
mnutt 9y ago
If you have multiple workers each managing their own throttling, there is a particular failure mode you may run into: everything is operating normally, then requests increase to the point that one of your servers starts throttling requests. Your load balancer redistributes traffic to the remaining machines, which see even greater load, and each in turn start throttling requests. At some point you have no working servers, and all requests are failing. You start adding servers, but as soon as a server enters the load balancer pool it is bombarded with all requests and shuts off. At this point you have more than enough servers to handle requests, but all of them are disabled. Most solutions to the thundering herd problem involve rate limiting at the load balancer, or a finer-grained heuristic to ensure that each server does as much work as it can, even under load.

by TekMol 9y ago

nkozyra 9y ago

But by this point the request has already been accepted and processed. It's also using a rolling average, which means you'll be "killing" connections that otherwise wouldn't need to be killed.

TekMol 9y ago
Well, there is no exact threshold what "needs to bee killed". The higher the load the longer it takes to process requests. At some point you rather want to drop additional requests instead of making things even slower for everybody. This is what happens here. Think of it like this: When the server is already darn slow at a load of 10 we don't want to process more requests because we know at values over 10 it gets unbearable.
- nkozyra 9y ago
  My point is it's still processing the request. You're just shielding the web server from whatever the application holds. In other words, let's find the theoretical limit to what your web server can handle in terms of concurrent requests. Let's throw them all at your application. In this scenario, the check-load-then-die code is worthless because the request will never get there. It doesn't solve the problem as shown in the OP. What it does is mitigate the effect of the application on the total load of the web server. Which is another problem altogether. Also: > drop additional requests instead of making things even slower for everybody. This is what happens here. Not exactly. The request completes like any HTTP request would. That you're die()ing out doesn't change that, at least from the perspective of the web server.
  - TekMol 9y ago
    find the theoretical limit to what your web server can handle in terms of concurrent requests. Let's throw them all at your application. In this scenario, the check-load-then-die code is worthless because the request will never get there. If there is such a limit, then why not mitigate that by setting a max number of connections for the webserver? You're just shielding the web server from whatever the application holds. True. But as I said: it works. Never seen our server get slow since we implemented this a few years ago. It kicks in when we get about 25x our normal traffic. Which happened less then once a year so far.

mnutt 9y ago

If you have multiple workers each managing their own throttling, there is a particular failure mode you may run into: everything is operating normally, then requests increase to the point that one of your servers starts throttling requests. Your load balancer redistributes traffic to the remaining machines, which see even greater load, and each in turn start throttling requests. At some point you have no working servers, and all requests are failing. You start adding servers, but as soon as a server enters the load balancer pool it is bombarded with all requests and shuts off. At this point you have more than enough servers to handle requests, but all of them are disabled. Most solutions to the thundering herd problem involve rate limiting at the load balancer, or a finer-grained heuristic to ensure that each server does as much work as it can, even under load.