4 ms·
Cloudflare, make sure you upgrade to 0.11.3, the new scheduling behavior is awesome for large clusters. Also a massive warning to anyone wanting to use hard cp
by chucky_z 6y ago
Cloudflare, make sure you upgrade to 0.11.3, the new scheduling behavior is awesome for large clusters.
Also a massive warning to anyone wanting to use hard cpu limits and cgroups (they do work in nomad it’s just not trivial), they don’t work like anyone expects and need to be heavily tested.
- waynesonfire 6y agoare you serious? you've got a major company touting a technology and a key component of it is broken?
- rbjorklin 6y agoI haven’t tested hard CPU quotas with Nomad but I suspect the issue mentioned above is due to cgroups/CFS and is also applicable to Kubernetes. See these slides for more details: https://www.slideshare.net/mobile/try_except_/optimizing-kubernetes-resource-requestslimits-for-costefficiency-and-latency-highload https://www.slideshare.net/mobile/try_except_/optimizing-kub...
- Pirate-of-SV 6y agoI recall that there's been some improvements to CFS to fix this since 2018.
- chucky_z 6y agoThere's been a ton of improvements over time. That doesn't make it any less foot-gun-y, unfortunately. I had the exact same problem with Kubernetes (and even just straight up containers), I just want to make sure folks are extremely aware of how big of a footgun it is, and to really, really, really test it well.
- chucky_z 6y agoYes, you know that small organization, Linux; who manages cgroups that everything else calls through with. I have only tested it with Docker, again, that company Moby with this very little used software. How dare their core systems have any systems that are not extremely simple to understand and have caveats.
- jjeaff 6y agoHa, have you ever tried kubernetes?
- aeyes 6y agoWhat problem did you observe regarding CPU limits? Is it latency due to CFS CPU period?