3 ms·
Boy, I hope the memory constraints for cgroups are getting better. I'm stuck on an ancient platform where cgroups are relatively new-ish (SuSE Linux Enterprise
by wyldfire 11y ago
Boy, I hope the memory constraints for cgroups are getting better. I'm stuck on an ancient platform where cgroups are relatively new-ish (SuSE Linux Enterprise 11). We hit a huge anti-feature where fs cache eviction from a cgroup-memory-constrained process happened in the foreground and caused astonishing latency. Sounds like it was the intended design. Hopefully it's evolving to no longer be limited like that.
- teraflop 11y agoI'm not sure what you're referring to as an "anti-feature". By design, if a process requests memory when it's already at its limit, the kernel will wait until more memory is available -- which could require a writeback, if the contents of the pagecache are dirty. Would you have preferred for the kernel to instead allow the process to exceed its memory limit, even if that means evicting something else outside the container? If so, why not just set the limit higher? Apologies if I'm misunderstanding something.
- wyldfire 11y agoSorry if some of this is already clear but I'll try and back up a bit and give more context. Linux uses greedy caching for filesystems and it's usually pretty awesome. But by merely opting in to be in a memory-constrained cgroup, it is (was?) no longer able to reap those pages in the background. Perhaps it's because the kernel daemons that would ordinarily do that don't have the right context. Or in order to have it they'd need to have one instance for each cgroup dir/context. Effectively what we found was that opting out of cgroups memory constraints meant that we would still cause the filesystem cache to get evicted in order to make room for our memory allocations but with significantly lower latency. I suspect that what might have been happening is that when cgroup-memory-bound, we faulted in a disk-backed page and that triggered logic that decided it needed to evict other pages in order to make room for this one. So far, so good. But then perhaps it got opportunistic and figured "since I can't know that I need to background-evict these pages I had better foreground-evict as many as I can now." This is all wild speculation on my part. But what was clear was that our process would suffer a multi-second stall while evicting pages. Disabling cgroups-memory constraints meant that we would not hit this same condition. More background: our application would read mapped files into memory, transform the data then write it out over sockets. The file-reading-task would read files whose size exceeds the total system memory. All files were opened read-only, the pages were never dirtied. There were no swap devices. I don't recall all the specifics but the gist we got from the distro was that "yeah, it works that way and it's not a bug." All I wanted to do was make it so that the greedy process could eat up no more than half of system memory with stuff like the fscache or dumb allocations. That would mean that other processes would be much less likely to stall when doing memory allocations.
- cm3 11y agoAnd I wish Sun had used a different (more widely accepted) license when open sourcing stuff, so that we would be using an enhanced version of Solaris Zones.
- mariuolo 11y agoDo you think it was by chance that CDDL is incompatible with GPL?
- cm3 11y agoI don't know, but I remember reading from the involved Sun people that this wasn't the plan. Bonwick and Cantrill I believe. In the end what matters is the result more than your intention, when it comes to legalese. Sun could have gotten back to its former glory, but the lawyers somehow decided, maybe knowing they'd like to sell the company, that they needed a new license (CDDL). Or, you know, we're all just speculating and have no idea anyway. The end result for the wider tech ecosystem is painful. Had BTRFS been liberally licensed, it might be shared just as the graphics DRM bits are between Linux and the BSDs, and we would have a more mature BTRFS, maybe...