3 ms·

> containers provide less security than people imagine Well sure, this heavily depends on the people you're talking to and what their imaginations are like. If

> containers provide less security than people imagine Well sure, this heavily depends on the people you're talking to and what their imaginations are like. If it's the "containers are lightweight VMs" crowd then sure -- that view is fundamentally wrong. > Linux control groups provide no meaningful resource isolation. So this is a pretty bold claim -- as far as I can see processes certainly get OOMkilled when they use more resources than allowed for by their cgroup. Do you have a link you could share? Is some fundamental part of cgroups (v1? v2?) broken in some way I haven't heard of up until now such that everyone has patched around it and it does what it says on the tin but despite the code in the kernel?

jeffbee 6y ago
The incompressible resources, including memory, have the best isolation story, but it is by no means perfect. A process in a given control group can easily cause external memory consumption that either escapes accounting altogether, or is charged to another control group. An example is the way kernel slabs are assigned to control groups. There were some patches in late 2020 from Facebook to help fix this, but I am not sure if they are merged and released yet. Those patches are also just better, not perfect. Another example is a control group that is chronically out of memory. It may page vigorously, which will have external effects on other control groups through unaccounted resources like nvme controller time, memory fragmentation, global vmscan, etc. A third popular way to abuse the resources of other containers is through networking. The way Linux handles network traffic is frankly hostile to proportional resource sharing. A control group with zero CPU quota can still easily cause external CPU time consumption.
- hardwaresofton 6y ago
  Thanks for explaining, while I still think what you said was a bit hyperbolic, I definitely didn't know about these... externalities, I guess you could call them? Learned something today.
  - jeffbee 6y ago
    I think of control groups as being adequate for what their inventor intended: approximate sharing of resources between non-antagonistic, first-party processes. Outside of that zone they can't be relied on for much of anything.

jeffbee 6y ago

The incompressible resources, including memory, have the best isolation story, but it is by no means perfect. A process in a given control group can easily cause external memory consumption that either escapes accounting altogether, or is charged to another control group. An example is the way kernel slabs are assigned to control groups. There were some patches in late 2020 from Facebook to help fix this, but I am not sure if they are merged and released yet. Those patches are also just better, not perfect. Another example is a control group that is chronically out of memory. It may page vigorously, which will have external effects on other control groups through unaccounted resources like nvme controller time, memory fragmentation, global vmscan, etc. A third popular way to abuse the resources of other containers is through networking. The way Linux handles network traffic is frankly hostile to proportional resource sharing. A control group with zero CPU quota can still easily cause external CPU time consumption.

hardwaresofton 6y ago
Thanks for explaining, while I still think what you said was a bit hyperbolic, I definitely didn't know about these... externalities, I guess you could call them? Learned something today.
- jeffbee 6y ago
  I think of control groups as being adequate for what their inventor intended: approximate sharing of resources between non-antagonistic, first-party processes. Outside of that zone they can't be relied on for much of anything.