3 ms·
It usually doesn't work that way. Usually, when your server is overloaded, instead of serving some fraction of the requests in reasonable time, it tries to queu
by barrkel 16y ago
It usually doesn't work that way. Usually, when your server is overloaded, instead of serving some fraction of the requests in reasonable time, it tries to queue the requests and serve them in order, and as a result response time goes through the roof. Often, the user at the other end of the causal chain starts getting frustrated and refreshes, adding more requests to the queue.
When servers are getting overloaded, you need to throttle the incoming requests aggressively and early, to keep response time reasonable.
- JoachimSchipper 16y agoIn network engineering, systems quite often use RED (http://en.wikipedia.org/wiki/Random_early_detection http://en.wikipedia.org/wiki/Random_early_detection) - proactively drop a percentage of connections/packets/... when loads begins to climb, scaling from 0% (any easily handled load) to 100% (at capacity). In practice, this results in pretty robust systems. (But note that TCP et al. have mechanisms that help this, e.g. slowing down when packets are lost and retrying a connection with a backoff.)