3 ms·
A classical solution is to use a control algorithm (for example PI or PID) to avoid having to estimate the overload threshold: the server will just start sheddi
by ot 9y ago
A classical solution is to use a control algorithm (for example PI or PID) to avoid having to estimate the overload threshold: the server will just start shedding a fraction of the requests to maintain some metric (CPU utilization, latency, queue length, ...) below some target.
See for example [1]. This is about background jobs, but the same principles apply to load shedding.
[1] http://folk.ntnu.no/skoge/prost/proceedings/acc04/Papers/0354_ThA01.3.pdf http://folk.ntnu.no/skoge/prost/proceedings/acc04/Papers/035...
- evanj 9y agoThanks for the link, the abstract seems relevant. I agree: a control algorithm seems like it should work. The challenge is figuring out the metrics and parameters to make something that works "well enough" for most applications, like TCP does for networks. I would really love for someone to figure that out, so we don't run into this very often.
- ot 9y agoYeah, it's something that needs to be tuned per-service. It is quite easy if the service is X-bound for some local resource X (CPU, disk, flash, network card), but if it is bottlenecked on external service calls it can be quite challenging to define a representative stress metric.