3 ms·
Not from stripe, but I've seen pretty bad CPU throttling. Often see quota's get exhausted through short bursts that don't show up in metrics that then causes C
by mnahkies 6y ago
Not from stripe, but I've seen pretty bad CPU throttling.
Often see quota's get exhausted through short bursts that don't show up in metrics that then causes CFS throttling to occur even though it looks like the pod is no where near its limit. Also struggled with application startup requiring far more CPU than at runtime leading to ridiculously slow startup times if you had a low limit.
So far our solution has been to just remove CPU limits, but hoping things will get better.
Removing the limits really improved our latency tail, and so far hasn't resulted in CPU saturation at the node level but your mileage may vary
- tbrock 6y agoDid you know that the Linux kernel has a bug that makes CPU limits for containers extra costly? https://github.com/kubernetes/kubernetes/issues/67577 https://github.com/kubernetes/kubernetes/issues/67577 https://github.com/torvalds/linux/commit/512ac999d2755d2b7109e996a76b6fb8b888631d https://github.com/torvalds/linux/commit/512ac999d2755d2b710... If I recall correctly you need 4.18+ to get the fix.
- dilyevsky 6y agoWe’ve seen just regular throttling too especially with erlang vm and go workloads