3 ms·
If you want to optimize for throughout one option is to overcommit resources (CPU/MEM/IO/NET) but utilize backpressure mechanisms to reduce load during times of
by __turbobrew__ 2y ago
If you want to optimize for throughout one option is to overcommit resources (CPU/MEM/IO/NET) but utilize backpressure mechanisms to reduce load during times of saturation.
Kubernetes does this through node pressure eviction but it is pretty easy to hook into the pressure stall information and have the application handle this as well (for example, start returning 429 HTTP responses when PSI goes over a certain level).
At the end of the day the optimal overcommit is workload dependent — good metrics and an iterative approach is needed.
- eru 2y agoYou probably also want to do somewhat intelligent load shedding, where you basically make sure you drop the most expendable items first. That's a basic fact of life for CPU cycles even for consumers on their machines, we call that just CPU priorities or so. But it's also useful for the other resources; but there it's a bit more disruptive, because we generally need to disrupt a program to eg claim back the disk space it uses.