5 ms·

> barco enforces a minimal set of restrictions to run untrusted code, which is not recommended for production use, where a more robust solution should be used.

> barco enforces a minimal set of restrictions to run untrusted code, which is not recommended for production use, where a more robust solution should be used. Aren't containers never suitable for running untrusted code? You need AppArmor, bwrap, or similar AFAIK.

loeg 3y ago
I would probably point at a virtual machine for a convenient place to run untrusted code. It's not perfect -- there are VM escapes -- but it's more convenient than a dedicated, air-gapped machine.
- CameronNemo 3y ago
  GKE runs every kubelet in its own gvisor-like userspace hypervisor. https://cloud.google.com/blog/products/containers-kubernetes/how-gvisor-protects-google-cloud-services-from-cve-2020-14386 https://cloud.google.com/blog/products/containers-kubernetes...
charcircuit 3y ago
>Aren't containers never suitable for running untrusted code? They are suitable provided the kernel is secure.
- cyphar 3y ago
  This is tautologically true -- "Is X secure? Yes, assuming the technology X uses is secure." The more nuanced answer is that containers have several layers of protections (seccomp, LSMs, user namespaces, namespaces, cgroups, capabilities, and standard process permissions by running as an unprivileged user) which all act together to help protect against container attacks. It's not perfect, but most container breakout attacks we've had so far are related to when container runtimes have to operate on a container during process setup (IMHO because the process for creating a container process is far from atomic) -- some of these attacks were enabled by kernel bugs which we went and fixed as well. It is very difficult to break out of a container once it has been configured and left alone.
  - raesene9 3y ago
    Possibly, but I'd say that Google's experience with kCTF was that allowing io_uring on hosts running containers has allowed for multiple breakouts. They paid out over $1m on io_uring related bug bounties https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html https://security.googleblog.com/2023/06/learnings-from-kctf-... Also while user namespaces help in theory, in practice expanding the attack surface of the kernel exposed to unprivileged users has consequences in allowing container breakout (e.g. CVE-2022-0185) also more recently CVE-2023-3390
    - cyphar 3y ago
      io_uring was blocked by the default Docker seccomp profile until somewhat recently. The primary issue with io_uring is that there is no mechanism to apply seccomp-like rules. > Also while user namespaces help in theory You should use user namespaces to contain untrusted code, you absolutely should not enable CLONE_NEWUSER inside a container. I was referring to the former, you're talking about the latter.
      - raesene9 3y ago
        for io_uring, I feel like it's a combination of bypassing seccomp and also the complexity of the code seems to be a fertile ground for Privesc vulnerabilities. I'd agree that unprivileged user namespaces should not be available inside containers, however the default stances of some Linux distros (enable unprivileged user namespaces) and Kubernetes (disable the CRIs seccomp filter by default) mean that an awful lot of envirionments will end up in a situation where this is possible.
cyphar 3y ago
bwrap is a container and AppArmor is used by basically every container runtime if the system is using AppArmor (otherwise they use SELinux). Seccomp is also enabled by default, and I would argue it is a more significant protection against container breakouts because it protects against kernel 0-days as well and doesn't rely on LSM hooks to block operations. The real question is whether you are using user namespaces. Jessica Frazelle ran a public bug bounty to break out of a container image that is properly secured, and as far as I know nobody collected the bounty. The website isn't up at the moment, maybe she took it down. https://contained.af/ https://contained.af/
- jppittma 3y ago
  Sounds like free money to me. You just press Ctrl+D, and you're out.
  - cyphar 3y ago
    Sadly that doesn't help you get access to the flag file you need to collect the bounty. ;)
viraptor 3y ago
Depends what you mean by suitable. If you run the service as a new user, it's more secure than running without a new namespace (you're isolated from other apps) and potentially less secure than running on host (one more layer of indirection for system resource access). Since in reality most attacks will be against your app itself before the attacker has direct access to syscalls, I see namespaces/containers as extra protection.
creatonez 3y ago
If built to spec, then the various container technologies in the kernel used together are theoretically secure. It closes all of the holes that we know about, aside from a few trivial things like the container spying on process id numbers on the host, and of course the vast potential to accidentally misconfigure it. However, all this code is quite complex, and the kernel and the software ecosystem are lacking in having a layered approach to security that goes all the way down to the low-level nitty gritty stuff. For example, kernel memory structures are not robustly protected against the usual memory exploits, and there isn't as strong W^X protection as desired. Windows, in contrast, is able to provide layered security through a variety of approaches, including running the entire operating system in a virtual machine, with the host ensuring integrity of kernel memory. These sorts of layered approaches to security are desirable because there will always be defects in any complex software. Side note: AppArmor and bwrap are distinct. Bubblewrap is a relatively simple userspace program that makes use of existing kernel containerization features (the same ones that Docker/Podman use), whereas AppArmor and SELinux are security features that are patched into the kernel itself. AppArmor and SELinux have made some progress in adding layered low-level security to the kernel, but it's not particularly impressive. Bubblewrap has done great work in exposing the kernel's existing tech to users, but they are not fundamental improvements to the kernel itself.
- viraptor 3y ago
  > aside from a few trivial things like the container spying on process id numbers on the host Containers with own PID namespace can't spy on process IDs on the host though? Not sure what you mean here. > and there isn't as strong W^X protection as desired What level is desired? Bootup warnings for W^X got merged a while ago. Changes that try to include anything violating it are rejected (see bcachefs). > Windows, in contrast, is able to provide layered security through a variety of approaches, including running the entire operating system in a virtual machine, with the host ensuring integrity of kernel memory. What? Xen existed for years, that's not "in contrast". Secureboot and lockdown exists on Linux too. There's also per-service firecracker microvm. > whereas AppArmor and SELinux are new security features that are patched into the kernel itself That's very misleading. They're not new - selinux is over 2 decades old. They're also not "patched in" - LSMs have been integrated into Linux for a very long time with multiple implementations available. Selinux had multilabel security created for gov use. It's quite impressive actually.

loeg 3y ago

I would probably point at a virtual machine for a convenient place to run untrusted code. It's not perfect -- there are VM escapes -- but it's more convenient than a dedicated, air-gapped machine.

CameronNemo 3y ago
GKE runs every kubelet in its own gvisor-like userspace hypervisor. https://cloud.google.com/blog/products/containers-kubernetes/how-gvisor-protects-google-cloud-services-from-cve-2020-14386 https://cloud.google.com/blog/products/containers-kubernetes...

charcircuit 3y ago

>Aren't containers never suitable for running untrusted code? They are suitable provided the kernel is secure.

cyphar 3y ago
This is tautologically true -- "Is X secure? Yes, assuming the technology X uses is secure." The more nuanced answer is that containers have several layers of protections (seccomp, LSMs, user namespaces, namespaces, cgroups, capabilities, and standard process permissions by running as an unprivileged user) which all act together to help protect against container attacks. It's not perfect, but most container breakout attacks we've had so far are related to when container runtimes have to operate on a container during process setup (IMHO because the process for creating a container process is far from atomic) -- some of these attacks were enabled by kernel bugs which we went and fixed as well. It is very difficult to break out of a container once it has been configured and left alone.
- raesene9 3y ago
  Possibly, but I'd say that Google's experience with kCTF was that allowing io_uring on hosts running containers has allowed for multiple breakouts. They paid out over $1m on io_uring related bug bounties https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html https://security.googleblog.com/2023/06/learnings-from-kctf-... Also while user namespaces help in theory, in practice expanding the attack surface of the kernel exposed to unprivileged users has consequences in allowing container breakout (e.g. CVE-2022-0185) also more recently CVE-2023-3390
  - cyphar 3y ago
    io_uring was blocked by the default Docker seccomp profile until somewhat recently. The primary issue with io_uring is that there is no mechanism to apply seccomp-like rules. > Also while user namespaces help in theory You should use user namespaces to contain untrusted code, you absolutely should not enable CLONE_NEWUSER inside a container. I was referring to the former, you're talking about the latter.
    - raesene9 3y ago
      for io_uring, I feel like it's a combination of bypassing seccomp and also the complexity of the code seems to be a fertile ground for Privesc vulnerabilities. I'd agree that unprivileged user namespaces should not be available inside containers, however the default stances of some Linux distros (enable unprivileged user namespaces) and Kubernetes (disable the CRIs seccomp filter by default) mean that an awful lot of envirionments will end up in a situation where this is possible.

cyphar 3y ago

bwrap is a container and AppArmor is used by basically every container runtime if the system is using AppArmor (otherwise they use SELinux). Seccomp is also enabled by default, and I would argue it is a more significant protection against container breakouts because it protects against kernel 0-days as well and doesn't rely on LSM hooks to block operations. The real question is whether you are using user namespaces. Jessica Frazelle ran a public bug bounty to break out of a container image that is properly secured, and as far as I know nobody collected the bounty. The website isn't up at the moment, maybe she took it down. https://contained.af/ https://contained.af/

jppittma 3y ago
Sounds like free money to me. You just press Ctrl+D, and you're out.
- cyphar 3y ago
  Sadly that doesn't help you get access to the flag file you need to collect the bounty. ;)

viraptor 3y ago

Depends what you mean by suitable. If you run the service as a new user, it's more secure than running without a new namespace (you're isolated from other apps) and potentially less secure than running on host (one more layer of indirection for system resource access). Since in reality most attacks will be against your app itself before the attacker has direct access to syscalls, I see namespaces/containers as extra protection.

creatonez 3y ago

If built to spec, then the various container technologies in the kernel used together are theoretically secure. It closes all of the holes that we know about, aside from a few trivial things like the container spying on process id numbers on the host, and of course the vast potential to accidentally misconfigure it. However, all this code is quite complex, and the kernel and the software ecosystem are lacking in having a layered approach to security that goes all the way down to the low-level nitty gritty stuff. For example, kernel memory structures are not robustly protected against the usual memory exploits, and there isn't as strong W^X protection as desired. Windows, in contrast, is able to provide layered security through a variety of approaches, including running the entire operating system in a virtual machine, with the host ensuring integrity of kernel memory. These sorts of layered approaches to security are desirable because there will always be defects in any complex software. Side note: AppArmor and bwrap are distinct. Bubblewrap is a relatively simple userspace program that makes use of existing kernel containerization features (the same ones that Docker/Podman use), whereas AppArmor and SELinux are security features that are patched into the kernel itself. AppArmor and SELinux have made some progress in adding layered low-level security to the kernel, but it's not particularly impressive. Bubblewrap has done great work in exposing the kernel's existing tech to users, but they are not fundamental improvements to the kernel itself.

viraptor 3y ago
> aside from a few trivial things like the container spying on process id numbers on the host Containers with own PID namespace can't spy on process IDs on the host though? Not sure what you mean here. > and there isn't as strong W^X protection as desired What level is desired? Bootup warnings for W^X got merged a while ago. Changes that try to include anything violating it are rejected (see bcachefs). > Windows, in contrast, is able to provide layered security through a variety of approaches, including running the entire operating system in a virtual machine, with the host ensuring integrity of kernel memory. What? Xen existed for years, that's not "in contrast". Secureboot and lockdown exists on Linux too. There's also per-service firecracker microvm. > whereas AppArmor and SELinux are new security features that are patched into the kernel itself That's very misleading. They're not new - selinux is over 2 decades old. They're also not "patched in" - LSMs have been integrated into Linux for a very long time with multiple implementations available. Selinux had multilabel security created for gov use. It's quite impressive actually.