5 ms·

Container Security with SELinux and CoreOS

44 points by jonaslejon 11y ago

jfindley 11y ago
Atomic[0] provides this on a Fedora/CentOS + docker base, and it works very well. It's slightly more mature than the CoreOS version as there is support for sharing volumes (volumes mounted with z are sharable, volumes mounted with Z are specific to a container). It's great to see this being adopted more widely - it greatly improves the security of containers, and in my experience, adds nothing to the cost of administrating systems. I'm much happier running our security-sensitive applications in containers with this than I would be without it. 0: http://www.projectatomic.io/ http://www.projectatomic.io/
- mjg59 11y ago
  Yeah, we're spending a little time exploring whether there's any feasible way to support sharing in a slightly more fine-grained manner - right now volumes shared between containers under Atomic are labeled such that all containers can write to them, which isn't entirely optimal.
benmmurphy 11y ago
'However, if the attacker is able to take advantage of a kernel bug, they may be able to escape from the container environment. ... every container runs in a different context and SELinux will forbid cross-context access, preventing any further damage from occurring.' In this scenario SELinux will only protect you from very specific kernel bugs. SELinux will not protect you against any kernel bug that either gives you a memory write primitive or gives you control over RIP. (Since you can just trash all the SELinux protections). It is only going to protect you from logic bugs where the kernel accidentally gives you filesystem access you weren't meant to have or something similar. Bare in mind that some of the recent kernel bugs relating to namespaces have been logic bugs.
- noja 11y ago
  Have you got any numbers? It would be interesting to find out what percentage of VM escape bugs have been protected by SELinux.
  - benmmurphy 11y ago
    It could be zero. I'm really not sure. Someone has to go through the vulns and check if they are protected or not. It probably doesn't harm security and may stop some future bug but if you have limited resources to apply to security it might be worth installing grsecurity instead of this. (https://grsecurity.net/ https://grsecurity.net/) They are often boasting about how grsecurity would have stopped a vuln. So they seem to have a track record of useful mitigations.
    - baghira 11y ago
      Selinux should block the VENOM vulnerability in Qemu: http://danwalsh.livejournal.com/71489.html http://danwalsh.livejournal.com/71489.html See also http://www.selinuxproject.org/page/SVirt http://www.selinuxproject.org/page/SVirt Keep in mind that Grsec (which is definitely underdeployed in the linux community) is compatible with LSM such as SElinx and AppArmor, so you do not have to choose (you'll take a performance hit, though).
      - benmmurphy 11y ago
        Do you know why people prefer to use labels in SELinux to isolate different Qemu instances rather than use different UIDs? Is it just a matter of convenience?
        mjg59 11y ago
        It's easy to grant a process the privilege to transition to isolated SELiniux contexts, but it's difficult to grant a process the privilege to transition to a limited range of UIDs. Doing it with SELinux also makes it easier to provide additional restrictions - there's a lot of files and device nodes that have to be a+r for normal processes, but which don't need to be available to code running in containers. Using SELinux allows you to prevent the container processes from having access to anything other than the files inside their namespace without needing to entirely refactor the permissions in the host.
raspasov 11y ago
Nice high level overview of container security via SELinux - anyone interested in running containers should read this. We are exploring different operating systems for running containers and CoreOS seems like a good candidate.
Klasiaster 11y ago
Does someone know how this would be manually applied to systemd-nspawn containers? Beside a kernel bug, what are the things a root process in a container could do to damage the host?
- andor 11y ago
  The systemd-nspawn man page has an example that might help: Example 6. Run a container with SELinux sandbox security contexts # chcon system_u:object_r:svirt_sandbox_file_t:s0:c0,c1 -R \ /srv/container # systemd-nspawn -L system_u:object_r:svirt_sandbox_file_t:s0:c0,c1 \ -Z system_u:system_r:svirt_lxc_net_t:s0:c0,c1 -D /srv/container \ /bin/sh In addition to -Z and -L, there's also --drop-capability. Apart from privilege escalation bugs, any shared resource can theoretically be used to escape containers. For example, access to main memory via a device with DMA, access to the same /proc as the host, access to a shared filesystem (replace a binary that the host executes), etc.
- SEJeff 11y ago
  The same thing a root user inside of a chroot can do... break out of the namespace and make it into the host system, as root. http://www.projectatomic.io/blog/2014/09/yet-another-reason-containers-don-t-contain-kernel-keyrings/ http://www.projectatomic.io/blog/2014/09/yet-another-reason-... http://opensource.com/business/14/7/docker-security-selinux http://opensource.com/business/14/7/docker-security-selinux
cakes 11y ago
I have been having a lot of conversations about SELinux + Containers recently. Is there more/further reading anyone can suggest about interactions/background between containers and SELinux (e.g. docker, systemd-nspawn, etc.)?

44 points by jonaslejon 11y ago

Atomic[0] provides this on a Fedora/CentOS + docker base, and it works very well. It's slightly more mature than the CoreOS version as there is support for sharing volumes (volumes mounted with z are sharable, volumes mounted with Z are specific to a container). It's great to see this being adopted more widely - it greatly improves the security of containers, and in my experience, adds nothing to the cost of administrating systems. I'm much happier running our security-sensitive applications in containers with this than I would be without it. 0: http://www.projectatomic.io/ http://www.projectatomic.io/

mjg59 11y ago
Yeah, we're spending a little time exploring whether there's any feasible way to support sharing in a slightly more fine-grained manner - right now volumes shared between containers under Atomic are labeled such that all containers can write to them, which isn't entirely optimal.

benmmurphy 11y ago

'However, if the attacker is able to take advantage of a kernel bug, they may be able to escape from the container environment. ... every container runs in a different context and SELinux will forbid cross-context access, preventing any further damage from occurring.' In this scenario SELinux will only protect you from very specific kernel bugs. SELinux will not protect you against any kernel bug that either gives you a memory write primitive or gives you control over RIP. (Since you can just trash all the SELinux protections). It is only going to protect you from logic bugs where the kernel accidentally gives you filesystem access you weren't meant to have or something similar. Bare in mind that some of the recent kernel bugs relating to namespaces have been logic bugs.

noja 11y ago
Have you got any numbers? It would be interesting to find out what percentage of VM escape bugs have been protected by SELinux.
- benmmurphy 11y ago
  It could be zero. I'm really not sure. Someone has to go through the vulns and check if they are protected or not. It probably doesn't harm security and may stop some future bug but if you have limited resources to apply to security it might be worth installing grsecurity instead of this. (https://grsecurity.net/ https://grsecurity.net/) They are often boasting about how grsecurity would have stopped a vuln. So they seem to have a track record of useful mitigations.
  - baghira 11y ago
    Selinux should block the VENOM vulnerability in Qemu: http://danwalsh.livejournal.com/71489.html http://danwalsh.livejournal.com/71489.html See also http://www.selinuxproject.org/page/SVirt http://www.selinuxproject.org/page/SVirt Keep in mind that Grsec (which is definitely underdeployed in the linux community) is compatible with LSM such as SElinx and AppArmor, so you do not have to choose (you'll take a performance hit, though).
    - benmmurphy 11y ago
      Do you know why people prefer to use labels in SELinux to isolate different Qemu instances rather than use different UIDs? Is it just a matter of convenience?
      - mjg59 11y ago
        It's easy to grant a process the privilege to transition to isolated SELiniux contexts, but it's difficult to grant a process the privilege to transition to a limited range of UIDs. Doing it with SELinux also makes it easier to provide additional restrictions - there's a lot of files and device nodes that have to be a+r for normal processes, but which don't need to be available to code running in containers. Using SELinux allows you to prevent the container processes from having access to anything other than the files inside their namespace without needing to entirely refactor the permissions in the host.

raspasov 11y ago

Nice high level overview of container security via SELinux - anyone interested in running containers should read this. We are exploring different operating systems for running containers and CoreOS seems like a good candidate.

Klasiaster 11y ago

Does someone know how this would be manually applied to systemd-nspawn containers? Beside a kernel bug, what are the things a root process in a container could do to damage the host?

andor 11y ago
The systemd-nspawn man page has an example that might help: Example 6. Run a container with SELinux sandbox security contexts # chcon system_u:object_r:svirt_sandbox_file_t:s0:c0,c1 -R \ /srv/container # systemd-nspawn -L system_u:object_r:svirt_sandbox_file_t:s0:c0,c1 \ -Z system_u:system_r:svirt_lxc_net_t:s0:c0,c1 -D /srv/container \ /bin/sh In addition to -Z and -L, there's also --drop-capability. Apart from privilege escalation bugs, any shared resource can theoretically be used to escape containers. For example, access to main memory via a device with DMA, access to the same /proc as the host, access to a shared filesystem (replace a binary that the host executes), etc.
SEJeff 11y ago
The same thing a root user inside of a chroot can do... break out of the namespace and make it into the host system, as root. http://www.projectatomic.io/blog/2014/09/yet-another-reason-containers-don-t-contain-kernel-keyrings/ http://www.projectatomic.io/blog/2014/09/yet-another-reason-... http://opensource.com/business/14/7/docker-security-selinux http://opensource.com/business/14/7/docker-security-selinux

cakes 11y ago

I have been having a lot of conversations about SELinux + Containers recently. Is there more/further reading anyone can suggest about interactions/background between containers and SELinux (e.g. docker, systemd-nspawn, etc.)?