3 ms·
Naively, you take N cpu-seconds to handle one request (say N=0.01). That means you can't handle more than 1/N requests/second before you start to slow down all
by jbert 17y ago
Naively, you take N cpu-seconds to handle one request (say N=0.01).
That means you can't handle more than 1/N requests/second before you start to slow down all requests.
So if you don't have a bound of 1/N requests/second on your server, you'll have a latency increase if the connection rate goes above that. At some pt that latency increase will be problematic.
It's also may be worse than the naive assessment above, since resources required to process one request might be flat but might go up as total #conns goes up (you'll hit a wall if your memory usage is enough to get into swap, you may be ok with an O(1) lookup from socket->connection state tho).
A decision by the server to refuse a TCP connect when it's at a configured limit (or dynamically determined limit) can be detected by a load balancer more easily than the latency on what might be varied requests.
Realworld example: people like to set MaxClients on apache, and for good reasons.
- axod 17y agoFWIW terrible example, that's why Apache is always blowing up when it gets much load. I'd much rather a server service all requests properly than fail. Then you can measure the load yourself, and load balance appropriately, knowing that all requests are being serviced properly.
- jbert 17y ago> I'd much rather a server service all requests properly than fail. Depends what you mean by "properly". If you mean "within N seconds", then when overload takes you over the point at which reqs take N seconds, all your reqs are now failing to be served properly. Whereas if you have a cap at M concurrent reqs (which you calculate so that your platform can service each of those M reqs within N secs), you get to handle M concurrent reqs properly under overload. So, not under overload - same behaviour. Under overload, a cap allows M concurrent reqs to be handled properly, no cap allows none. What alternative are you suggesting? Let the platform run slower as load increases until your alarm system pings you at a threshold and you then provision more servers?