7 ms·

A good summary of interesting new things in this release can be found at http://kernelnewbies.org/Linux_4.6 http://kernelnewbies.org/Linux_4.6

dominotw 10y ago
>Support for cgroup namespaces This release adds support for cgroup namespaces, which provides a mechanism to virtualize the view of the /proc/$PID/cgroup file and cgroup mounts. What are the advantages and practical applications of this ? documentation here : https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d4021f6cd41f03017f831b3d40b0067bed54893d https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux....
- webkike 10y ago
  I imagine all the advantages that come with namespaces in a more advanced OS like plan 9
  - zxcvcxz 10y ago
    How many large companies are using plan 9 in production though?
    - gh02t 10y ago
      To a first approximation... zero. That is to say, I don't know any (and I've looked a bit) and the realities of the limited hardware support make it difficult to imagine. I don't know of anybody using Inferno either, but I'm less familiar with it. I'd also dispute the suggestion that Plan9 is more advanced than Linux. It's different and Plan9 definitely does some interesting things, but that doesn't automatically make it more advanced. Ignoring that "advanced" is hard to quantify, I'd more that Plan9 is simply based on different concepts. It's like comparing Windows and Linux, albeit maybe not quite so extreme as Plan9 and Linux have more in common and Plan9 is simply not as heavily developed as either OS.
    - cm3 10y ago
      Coraid used to build and sell systems based on Plan9, but I believe they don't exist anymore, though I don't mean to imply Plan9 was at fault, quite the contrary.
    - stonogo 10y ago
      not only is this a pretty stupid measuring stick, this non sequitur application of it comes off as needlessly defensive
  - stormbrew 10y ago
    Don't think this has anything to do with anything plan9 can do that Linux can't already. The barriers to enabling the magic plan9 has are more along the lines of making it feasible for unprivileged users to mount and create namespaces themselves, which is complicated (afaik) largely by the way privilege escalation happens in unix systems. This just looks like it prevents some information from leaking out about the host system from inside a cgroup by inspecting the /proc/PID/cgroup file.
- wyldfire 10y ago
  Boy, I hope the memory constraints for cgroups are getting better. I'm stuck on an ancient platform where cgroups are relatively new-ish (SuSE Linux Enterprise 11). We hit a huge anti-feature where fs cache eviction from a cgroup-memory-constrained process happened in the foreground and caused astonishing latency. Sounds like it was the intended design. Hopefully it's evolving to no longer be limited like that.
  - teraflop 10y ago
    I'm not sure what you're referring to as an "anti-feature". By design, if a process requests memory when it's already at its limit, the kernel will wait until more memory is available -- which could require a writeback, if the contents of the pagecache are dirty. Would you have preferred for the kernel to instead allow the process to exceed its memory limit, even if that means evicting something else outside the container? If so, why not just set the limit higher? Apologies if I'm misunderstanding something.
    - wyldfire 10y ago
      Sorry if some of this is already clear but I'll try and back up a bit and give more context. Linux uses greedy caching for filesystems and it's usually pretty awesome. But by merely opting in to be in a memory-constrained cgroup, it is (was?) no longer able to reap those pages in the background. Perhaps it's because the kernel daemons that would ordinarily do that don't have the right context. Or in order to have it they'd need to have one instance for each cgroup dir/context. Effectively what we found was that opting out of cgroups memory constraints meant that we would still cause the filesystem cache to get evicted in order to make room for our memory allocations but with significantly lower latency. I suspect that what might have been happening is that when cgroup-memory-bound, we faulted in a disk-backed page and that triggered logic that decided it needed to evict other pages in order to make room for this one. So far, so good. But then perhaps it got opportunistic and figured "since I can't know that I need to background-evict these pages I had better foreground-evict as many as I can now." This is all wild speculation on my part. But what was clear was that our process would suffer a multi-second stall while evicting pages. Disabling cgroups-memory constraints meant that we would not hit this same condition. More background: our application would read mapped files into memory, transform the data then write it out over sockets. The file-reading-task would read files whose size exceeds the total system memory. All files were opened read-only, the pages were never dirtied. There were no swap devices. I don't recall all the specifics but the gist we got from the distro was that "yeah, it works that way and it's not a bug." All I wanted to do was make it so that the greedy process could eat up no more than half of system memory with stuff like the fscache or dumb allocations. That would mean that other processes would be much less likely to stall when doing memory allocations.
- cyphar 10y ago
  It also allows you to mount the cgroupfs in an unprivileged user namespace (useful for rootless containers, something I'm working on in runC). It's not entirely useful at the moment to be honest, since all it does is mask /proc/self/cgroup. But it could be extended later to also virtualise /proc/meminfo and /proc/cpuinfo to reflect the cgroup limits. I'm currently trying to get a patch merged into cgroup core to allow for processes in a cgroup namespace to manage their own cgroups (without compromising the hierarchy limits). Hopefully it gets into 4.7. This will be exceptionally useful for cgroup hierarchies.
- digi_owl 10y ago
  My impression i enhancing security within containers. This because now a would be attacker can't as easily tell if they are inside a container or not after exploiting some daemon vulnerability.
ZenoArrow 10y ago
Thanks for the link. I was surprised to learn that the BATMAN protocol had reached mainline Linux, and version 5 no less. I hadn't realised quite how healthy the development of this protocol was. https://en.wikipedia.org/wiki/B.A.T.M.A.N https://en.wikipedia.org/wiki/B.A.T.M.A.N. https://www.open-mesh.org/projects/open-mesh/wiki https://www.open-mesh.org/projects/open-mesh/wiki
reactor 10y ago
May be a noob question, but is Linus himself review all these commits before merging to mainline? Or each section (for eg: networking) got its own owners?
- ianleeclark 10y ago
  He has a system of trusted maintainers for different sub-sections of the kernel. There's simply too many patches for one man to review.

dominotw 10y ago

>Support for cgroup namespaces This release adds support for cgroup namespaces, which provides a mechanism to virtualize the view of the /proc/$PID/cgroup file and cgroup mounts. What are the advantages and practical applications of this ? documentation here : https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d4021f6cd41f03017f831b3d40b0067bed54893d https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux....

webkike 10y ago
I imagine all the advantages that come with namespaces in a more advanced OS like plan 9
- zxcvcxz 10y ago
  How many large companies are using plan 9 in production though?
  - gh02t 10y ago
    To a first approximation... zero. That is to say, I don't know any (and I've looked a bit) and the realities of the limited hardware support make it difficult to imagine. I don't know of anybody using Inferno either, but I'm less familiar with it. I'd also dispute the suggestion that Plan9 is more advanced than Linux. It's different and Plan9 definitely does some interesting things, but that doesn't automatically make it more advanced. Ignoring that "advanced" is hard to quantify, I'd more that Plan9 is simply based on different concepts. It's like comparing Windows and Linux, albeit maybe not quite so extreme as Plan9 and Linux have more in common and Plan9 is simply not as heavily developed as either OS.
  - cm3 10y ago
    Coraid used to build and sell systems based on Plan9, but I believe they don't exist anymore, though I don't mean to imply Plan9 was at fault, quite the contrary.
  - stonogo 10y ago
    not only is this a pretty stupid measuring stick, this non sequitur application of it comes off as needlessly defensive
- stormbrew 10y ago
  Don't think this has anything to do with anything plan9 can do that Linux can't already. The barriers to enabling the magic plan9 has are more along the lines of making it feasible for unprivileged users to mount and create namespaces themselves, which is complicated (afaik) largely by the way privilege escalation happens in unix systems. This just looks like it prevents some information from leaking out about the host system from inside a cgroup by inspecting the /proc/PID/cgroup file.
wyldfire 10y ago
Boy, I hope the memory constraints for cgroups are getting better. I'm stuck on an ancient platform where cgroups are relatively new-ish (SuSE Linux Enterprise 11). We hit a huge anti-feature where fs cache eviction from a cgroup-memory-constrained process happened in the foreground and caused astonishing latency. Sounds like it was the intended design. Hopefully it's evolving to no longer be limited like that.
- teraflop 10y ago
  I'm not sure what you're referring to as an "anti-feature". By design, if a process requests memory when it's already at its limit, the kernel will wait until more memory is available -- which could require a writeback, if the contents of the pagecache are dirty. Would you have preferred for the kernel to instead allow the process to exceed its memory limit, even if that means evicting something else outside the container? If so, why not just set the limit higher? Apologies if I'm misunderstanding something.
  - wyldfire 10y ago
    Sorry if some of this is already clear but I'll try and back up a bit and give more context. Linux uses greedy caching for filesystems and it's usually pretty awesome. But by merely opting in to be in a memory-constrained cgroup, it is (was?) no longer able to reap those pages in the background. Perhaps it's because the kernel daemons that would ordinarily do that don't have the right context. Or in order to have it they'd need to have one instance for each cgroup dir/context. Effectively what we found was that opting out of cgroups memory constraints meant that we would still cause the filesystem cache to get evicted in order to make room for our memory allocations but with significantly lower latency. I suspect that what might have been happening is that when cgroup-memory-bound, we faulted in a disk-backed page and that triggered logic that decided it needed to evict other pages in order to make room for this one. So far, so good. But then perhaps it got opportunistic and figured "since I can't know that I need to background-evict these pages I had better foreground-evict as many as I can now." This is all wild speculation on my part. But what was clear was that our process would suffer a multi-second stall while evicting pages. Disabling cgroups-memory constraints meant that we would not hit this same condition. More background: our application would read mapped files into memory, transform the data then write it out over sockets. The file-reading-task would read files whose size exceeds the total system memory. All files were opened read-only, the pages were never dirtied. There were no swap devices. I don't recall all the specifics but the gist we got from the distro was that "yeah, it works that way and it's not a bug." All I wanted to do was make it so that the greedy process could eat up no more than half of system memory with stuff like the fscache or dumb allocations. That would mean that other processes would be much less likely to stall when doing memory allocations.
cyphar 10y ago
It also allows you to mount the cgroupfs in an unprivileged user namespace (useful for rootless containers, something I'm working on in runC). It's not entirely useful at the moment to be honest, since all it does is mask /proc/self/cgroup. But it could be extended later to also virtualise /proc/meminfo and /proc/cpuinfo to reflect the cgroup limits. I'm currently trying to get a patch merged into cgroup core to allow for processes in a cgroup namespace to manage their own cgroups (without compromising the hierarchy limits). Hopefully it gets into 4.7. This will be exceptionally useful for cgroup hierarchies.
digi_owl 10y ago
My impression i enhancing security within containers. This because now a would be attacker can't as easily tell if they are inside a container or not after exploiting some daemon vulnerability.

ZenoArrow 10y ago

Thanks for the link. I was surprised to learn that the BATMAN protocol had reached mainline Linux, and version 5 no less. I hadn't realised quite how healthy the development of this protocol was. https://en.wikipedia.org/wiki/B.A.T.M.A.N https://en.wikipedia.org/wiki/B.A.T.M.A.N. https://www.open-mesh.org/projects/open-mesh/wiki https://www.open-mesh.org/projects/open-mesh/wiki

reactor 10y ago

May be a noob question, but is Linus himself review all these commits before merging to mainline? Or each section (for eg: networking) got its own owners?

ianleeclark 10y ago
He has a system of trusted maintainers for different sub-sections of the kernel. There's simply too many patches for one man to review.