4 ms·

I'm not sure what you're referring to as an "anti-feature". By design, if a process requests memory when it's already at its limit, the kernel will wait until m

I'm not sure what you're referring to as an "anti-feature". By design, if a process requests memory when it's already at its limit, the kernel will wait until more memory is available -- which could require a writeback, if the contents of the pagecache are dirty. Would you have preferred for the kernel to instead allow the process to exceed its memory limit, even if that means evicting something else outside the container? If so, why not just set the limit higher? Apologies if I'm misunderstanding something.

wyldfire 10y ago
Sorry if some of this is already clear but I'll try and back up a bit and give more context. Linux uses greedy caching for filesystems and it's usually pretty awesome. But by merely opting in to be in a memory-constrained cgroup, it is (was?) no longer able to reap those pages in the background. Perhaps it's because the kernel daemons that would ordinarily do that don't have the right context. Or in order to have it they'd need to have one instance for each cgroup dir/context. Effectively what we found was that opting out of cgroups memory constraints meant that we would still cause the filesystem cache to get evicted in order to make room for our memory allocations but with significantly lower latency. I suspect that what might have been happening is that when cgroup-memory-bound, we faulted in a disk-backed page and that triggered logic that decided it needed to evict other pages in order to make room for this one. So far, so good. But then perhaps it got opportunistic and figured "since I can't know that I need to background-evict these pages I had better foreground-evict as many as I can now." This is all wild speculation on my part. But what was clear was that our process would suffer a multi-second stall while evicting pages. Disabling cgroups-memory constraints meant that we would not hit this same condition. More background: our application would read mapped files into memory, transform the data then write it out over sockets. The file-reading-task would read files whose size exceeds the total system memory. All files were opened read-only, the pages were never dirtied. There were no swap devices. I don't recall all the specifics but the gist we got from the distro was that "yeah, it works that way and it's not a bug." All I wanted to do was make it so that the greedy process could eat up no more than half of system memory with stuff like the fscache or dumb allocations. That would mean that other processes would be much less likely to stall when doing memory allocations.

wyldfire 10y ago

Sorry if some of this is already clear but I'll try and back up a bit and give more context. Linux uses greedy caching for filesystems and it's usually pretty awesome. But by merely opting in to be in a memory-constrained cgroup, it is (was?) no longer able to reap those pages in the background. Perhaps it's because the kernel daemons that would ordinarily do that don't have the right context. Or in order to have it they'd need to have one instance for each cgroup dir/context. Effectively what we found was that opting out of cgroups memory constraints meant that we would still cause the filesystem cache to get evicted in order to make room for our memory allocations but with significantly lower latency. I suspect that what might have been happening is that when cgroup-memory-bound, we faulted in a disk-backed page and that triggered logic that decided it needed to evict other pages in order to make room for this one. So far, so good. But then perhaps it got opportunistic and figured "since I can't know that I need to background-evict these pages I had better foreground-evict as many as I can now." This is all wild speculation on my part. But what was clear was that our process would suffer a multi-second stall while evicting pages. Disabling cgroups-memory constraints meant that we would not hit this same condition. More background: our application would read mapped files into memory, transform the data then write it out over sockets. The file-reading-task would read files whose size exceeds the total system memory. All files were opened read-only, the pages were never dirtied. There were no swap devices. I don't recall all the specifics but the gist we got from the distro was that "yeah, it works that way and it's not a bug." All I wanted to do was make it so that the greedy process could eat up no more than half of system memory with stuff like the fscache or dumb allocations. That would mean that other processes would be much less likely to stall when doing memory allocations.