4 ms·

I guess, if you thought Docker/etc. was a security boundary

markasoftware 3mo ago
Runpod, digital ocean's gpu cloud, and at least a few others use Linux containers for isolation between tenants (look at Wiz's blog post about the nvidia container toolkit bug; digitalocean just puts everyone in a massive k8s cluster)
- stingraycharles 3mo ago
  Why aren’t they using a fast VM like Firecracker?
  - himata4113 3mo ago
    To squeeze out 5% more profit.
circularfoyers 3mo ago
I know there's a lot you can do in k8s to mitigate it, but I didn't think that prevented it outright.
insanitybit 3mo ago
They are a security boundary. The fact that you need a vulnerability to escape them is proof of that. They just don't have a particularly high cost of escape because reachable kernel vulnerabilities are so common.
- worthless-trash 3mo ago
  Some people clearly do use containers as deployment mechanism, with security not in mind.
  - zbentley 3mo ago
    s/some/most/ That's not meant to be snide, just true, I think.
    - CodesInChaos 3mo ago
      I never understood why kubernetes doesn't use a VM-per-pod model by default.
- dijit 3mo ago
  Escape from docker containers is trivially easy, if you are able to run as the root user in the container itself. Many (maybe most) containers actually default to running programs as root. Kernel exploit not required.
  - maple3142 3mo ago
    If you are given a shell with `docker run -it --rm alpine:3 sh`, can you read the /etc/shadow on the host without kernel exploit? Assuming the docker and kernel are sufficiently update-to-date (e.g. latest Docker on Debian Stable).
    - chlorion 3mo ago
      No. The "root" you get in docker is not actually root outside of the namespace the container in running in. Assuming no bugs in the kernel, it should not be able to do anything more than the UID that it's mapped from.
      - ptx 3mo ago
        Does Docker use user namespaces by default? Otherwise root in the container is actually root on the host, from what I read. Correct me if I'm wrong. (Privileges are still limited by seccomp filters blocking some syscalls, and there's SELinux to block some other stuff, but it's still the actual root user without user namespaces, I think?)
        insanitybit 3mo ago
        That's right. Docker still runs without user namespaces by default, which means that root is the same user inside and outside of the container. This does open up attack surface and configuration footguns. Confinement still leverages dropping some root caps, seccomp, various other namespaces, etc.
        inigyou 3mo ago
- password4321 3mo ago
  > They are a security boundary My mistake, leaving out some adjective one could interpret as a misunderstanding of containers as an effective (etc.) security boundary. Fool me 100+ times and all that. There must be at least a triple-digit number of CVEs by now demonstratimg that in practice containers are a thinner layer of security (perhaps not quite as thin as the classic recommendation of running SSH on a nonstandard port, but that might be leaning toward the safer side of analogies vs. malicious code!) rather than a boundary like virtualization (not perfect but a best practice for isolation).
XorNot 3mo ago
There was a virtual machine KVM escape found like 2 weeks ago. Nothing is a security boundary anymore.

Runpod, digital ocean's gpu cloud, and at least a few others use Linux containers for isolation between tenants (look at Wiz's blog post about the nvidia container toolkit bug; digitalocean just puts everyone in a massive k8s cluster)

stingraycharles 3mo ago
Why aren’t they using a fast VM like Firecracker?
- himata4113 3mo ago
  To squeeze out 5% more profit.

circularfoyers 3mo ago

I know there's a lot you can do in k8s to mitigate it, but I didn't think that prevented it outright.

insanitybit 3mo ago

They are a security boundary. The fact that you need a vulnerability to escape them is proof of that. They just don't have a particularly high cost of escape because reachable kernel vulnerabilities are so common.

worthless-trash 3mo ago
Some people clearly do use containers as deployment mechanism, with security not in mind.
- zbentley 3mo ago
  s/some/most/ That's not meant to be snide, just true, I think.
  - CodesInChaos 3mo ago
    I never understood why kubernetes doesn't use a VM-per-pod model by default.
dijit 3mo ago
Escape from docker containers is trivially easy, if you are able to run as the root user in the container itself. Many (maybe most) containers actually default to running programs as root. Kernel exploit not required.
- maple3142 3mo ago
  If you are given a shell with `docker run -it --rm alpine:3 sh`, can you read the /etc/shadow on the host without kernel exploit? Assuming the docker and kernel are sufficiently update-to-date (e.g. latest Docker on Debian Stable).
  - chlorion 3mo ago
    No. The "root" you get in docker is not actually root outside of the namespace the container in running in. Assuming no bugs in the kernel, it should not be able to do anything more than the UID that it's mapped from.
    - ptx 3mo ago
      Does Docker use user namespaces by default? Otherwise root in the container is actually root on the host, from what I read. Correct me if I'm wrong. (Privileges are still limited by seccomp filters blocking some syscalls, and there's SELinux to block some other stuff, but it's still the actual root user without user namespaces, I think?)
      - insanitybit 3mo ago
        That's right. Docker still runs without user namespaces by default, which means that root is the same user inside and outside of the container. This does open up attack surface and configuration footguns. Confinement still leverages dropping some root caps, seccomp, various other namespaces, etc.
        inigyou 3mo ago
password4321 3mo ago
> They are a security boundary My mistake, leaving out some adjective one could interpret as a misunderstanding of containers as an effective (etc.) security boundary. Fool me 100+ times and all that. There must be at least a triple-digit number of CVEs by now demonstratimg that in practice containers are a thinner layer of security (perhaps not quite as thin as the classic recommendation of running SSH on a nonstandard port, but that might be leaning toward the safer side of analogies vs. malicious code!) rather than a boundary like virtualization (not perfect but a best practice for isolation).

XorNot 3mo ago

There was a virtual machine KVM escape found like 2 weeks ago. Nothing is a security boundary anymore.