4 ms·

> Containers are not a security measure. They're acceptable for some resource constraint considerations, but you should not ever use them because of security co

> Containers are not a security measure. They're acceptable for some resource constraint considerations, but you should not ever use them because of security concerns. If that's something to worry about, use VMs at the very least. This is repeatedly really commonly, but is bordering on wrong more and more every year. At the lowest level it's not even completely correct. Containers are resource + namespace isolation -- cgroups and namespaces -- isolation and kernel-assisted resource hiding/access control is absolutely a facet of good defence-in-depth. Enhanced security is not a core feature of containerization as a whole but it can absolutely help with your security posture. For what % that satement is wrong, it's even more so because it fails to consider the container runtime. Container runtimes these days (containerd is the best one IMO) include ways to run containers at different levels of isolation (VMs and as micro kernels), so a move to containers can absolutely help security posture, because now you don't have to pull out packer/ansible/etc to build your VM, you can just take the same container you were running and change the runtime. Time to ship/launch improved security features is important, and containers can reduce that burden for teams. Containers are a process isolation and resource control tool -- they can absolutely aid in security posture. Would you say the same about BSD Jails? Just so there's some more value here, here are some projects that are somewhat close to cutting edge (though it's been a while) in this space that I like: - nabla-containers.github.io/ - https://github.com/firecracker-microvm/firecracker-containerd https://github.com/firecracker-microvm/firecracker-container... - https://github.com/kata-containers/runtime https://github.com/kata-containers/runtime (see https://github.com/kata-containers/documentation/blob/master/how-to/containerd-kata.md https://github.com/kata-containers/documentation/blob/master...) - https://rootlesscontaine.rs/ https://rootlesscontaine.rs/ - https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2... - https://starkandwayne.com/blog/the-capable-kernel-an-introduction-to-linux-capabilities/ https://starkandwayne.com/blog/the-capable-kernel-an-introdu... The container ecosystem surfaces a lot of really cool resource limiting and isolation techniques and puts them in very close reach.

jeffbee 6y ago
I’ll double down on what the GP said: containers provide less security than people imagine, and contrary to your comments Linux control groups provide no meaningful resource isolation.
- hardwaresofton 6y ago
  > containers provide less security than people imagine Well sure, this heavily depends on the people you're talking to and what their imaginations are like. If it's the "containers are lightweight VMs" crowd then sure -- that view is fundamentally wrong. > Linux control groups provide no meaningful resource isolation. So this is a pretty bold claim -- as far as I can see processes certainly get OOMkilled when they use more resources than allowed for by their cgroup. Do you have a link you could share? Is some fundamental part of cgroups (v1? v2?) broken in some way I haven't heard of up until now such that everyone has patched around it and it does what it says on the tin but despite the code in the kernel?
  - jeffbee 6y ago
    The incompressible resources, including memory, have the best isolation story, but it is by no means perfect. A process in a given control group can easily cause external memory consumption that either escapes accounting altogether, or is charged to another control group. An example is the way kernel slabs are assigned to control groups. There were some patches in late 2020 from Facebook to help fix this, but I am not sure if they are merged and released yet. Those patches are also just better, not perfect. Another example is a control group that is chronically out of memory. It may page vigorously, which will have external effects on other control groups through unaccounted resources like nvme controller time, memory fragmentation, global vmscan, etc. A third popular way to abuse the resources of other containers is through networking. The way Linux handles network traffic is frankly hostile to proportional resource sharing. A control group with zero CPU quota can still easily cause external CPU time consumption.
    - hardwaresofton 6y ago
      Thanks for explaining, while I still think what you said was a bit hyperbolic, I definitely didn't know about these... externalities, I guess you could call them? Learned something today.
      - jeffbee 6y ago
dundarious 6y ago
GP is noting the security limitations of Linux containers, the mechanism, not some management daemon like containerd or some API or workflow. firecracker-containerd is not a container, per-se, as stated in their README. > Like traditional containers, Firecracker microVMs offer fast start-up and shut-down and minimal overhead. Unlike traditional containers, however, they can provide an additional layer of isolation via the KVM hypervisor. I think the security mechanism of note there is the hypervisor -- a VM technology. The fact that it is manageable via a daemon/API that was created for containers, does not mean it provides security via Linux containers, the mechanism. A similar story can be told regarding capabilities, a mechanism that is orthogonal to containers -- all combinations of with and without capabilities and containers have their uses. I do think GP is overly glib and assumes a particular threat model in order to dismiss Linux containers so thoroughly, and you are right to note "they can absolutely aid in security posture", etc. But GP is correct to note that VMs (though also imperfect) provide certain kinds of security/isolation that Linux containers, the mechanism, cannot yet match.

jeffbee 6y ago

I’ll double down on what the GP said: containers provide less security than people imagine, and contrary to your comments Linux control groups provide no meaningful resource isolation.

hardwaresofton 6y ago
> containers provide less security than people imagine Well sure, this heavily depends on the people you're talking to and what their imaginations are like. If it's the "containers are lightweight VMs" crowd then sure -- that view is fundamentally wrong. > Linux control groups provide no meaningful resource isolation. So this is a pretty bold claim -- as far as I can see processes certainly get OOMkilled when they use more resources than allowed for by their cgroup. Do you have a link you could share? Is some fundamental part of cgroups (v1? v2?) broken in some way I haven't heard of up until now such that everyone has patched around it and it does what it says on the tin but despite the code in the kernel?
- jeffbee 6y ago
  The incompressible resources, including memory, have the best isolation story, but it is by no means perfect. A process in a given control group can easily cause external memory consumption that either escapes accounting altogether, or is charged to another control group. An example is the way kernel slabs are assigned to control groups. There were some patches in late 2020 from Facebook to help fix this, but I am not sure if they are merged and released yet. Those patches are also just better, not perfect. Another example is a control group that is chronically out of memory. It may page vigorously, which will have external effects on other control groups through unaccounted resources like nvme controller time, memory fragmentation, global vmscan, etc. A third popular way to abuse the resources of other containers is through networking. The way Linux handles network traffic is frankly hostile to proportional resource sharing. A control group with zero CPU quota can still easily cause external CPU time consumption.
  - hardwaresofton 6y ago
    Thanks for explaining, while I still think what you said was a bit hyperbolic, I definitely didn't know about these... externalities, I guess you could call them? Learned something today.
    - jeffbee 6y ago

dundarious 6y ago

GP is noting the security limitations of Linux containers, the mechanism, not some management daemon like containerd or some API or workflow. firecracker-containerd is not a container, per-se, as stated in their README. > Like traditional containers, Firecracker microVMs offer fast start-up and shut-down and minimal overhead. Unlike traditional containers, however, they can provide an additional layer of isolation via the KVM hypervisor. I think the security mechanism of note there is the hypervisor -- a VM technology. The fact that it is manageable via a daemon/API that was created for containers, does not mean it provides security via Linux containers, the mechanism. A similar story can be told regarding capabilities, a mechanism that is orthogonal to containers -- all combinations of with and without capabilities and containers have their uses. I do think GP is overly glib and assumes a particular threat model in order to dismiss Linux containers so thoroughly, and you are right to note "they can absolutely aid in security posture", etc. But GP is correct to note that VMs (though also imperfect) provide certain kinds of security/isolation that Linux containers, the mechanism, cannot yet match.