4 ms·

Why? Both are supposed to keep whatever is inside trapped unless you poke holes in that protection (say, using virtio or even just 9P to hand it real storage)

lmm 3y ago
Because virtual machines were designed from the start to run untrusted code, and containers were not? > supposed to keep whatever is inside trapped unless you poke holes in that protection As far as I know that was never a design decision for containers on Linux; certainly not in the early days.
- zekrioca 3y ago
  This doesn’t answer the question. How were VMs designed with security in mind and not with host emulation? Untrusted code? I’m confused, people are talking about vulnerabilities.
  - icedchai 3y ago
    VMs don’t share the kernel with the host. Any host escape would need to happen through a device driver exposed to the guest (virtio, etc.) Containers use the same kernel, obviously a much larger set of code. More code means a greater chance of vulnerabilities.
    - zekrioca 3y ago
      How do you think the VM itself is spawned? Something in the host instantiate it. But even if we ignore that, I would argue that concentrating in one location only (e.g., some exposed driver) to escape is also easier, since an attacker would need to spend time trying to find only one vulnerability rather than several. I think the hardware can help bridging the gap between containers and VMs by enabling userspace processes behave as VMs, which is more or less what QEMU+KVM try to do, except that it still comes with some overheads and less flexibility.
      - icedchai 3y ago
        I have to disagree. With a VM, there is less shared code shared with the host to review and audit for vulnerabilities. The developers can go over those device drivers, system calls for VM management, etc. with a fined-tooth comb. With containers, you have essentially the entire kernel, much more surface area to potentially exploit. Maybe I am wrong. We can wait for a security professional to comment.
  - dharmab 3y ago
    When you switch between VMs and the host the CPU executes an instruction to isolate the data of each VMs entirely (flushing caches). This doesn't happen with containers.
    - zekrioca 3y ago
      Although this feature does improve security, it is due to the CPU, the VM and the main idea was to reduce performance overheads. I do not see adding layers and layers of abstractions the same as more secure. In fact, meltdown and spectre serve as counter-examples. Also, one can achieve similar effects with containers as well, just think AppArmor, capabilities, permissions, etc., all layers of administrative privileges between some untrusted code and the host.
      - dharmab 3y ago
        There is nothing you can do in the OS that replaces the VMENTER instruction. You need this precisely because it mitigates SPECTRE and Meltdown, assuming your microcode is up to date.
- yjftsjthsd-h 3y ago
  I'm willing to believe that Linux containers were not initially designed to be a security boundary, but I struggle to see why that means they aren't now; it's been over a decade and they have an awful lot of security features for something that doesn't care about security. EDIT: For that matter, they're clearly being used for security; the features in Linux that are used by runc et al. are the same features used by eg. Chrome to isolate components in order to contain vulnerabilities.
  - fbdab103 3y ago
    As far as I know, Docker still punches through your firewall by default. I consider that a pretty big negative against assuming secure-by-design.
    - yjftsjthsd-h 3y ago
      What attack are you envisioning that is aided by docker bypassing firewall rules?
      - fbdab103 3y ago
        I mean, there was this story[0] ("How a Docker footgun led to a vandal deleting NewsBlur's MongoDB database") about how the Docker rules allowed a hacker to delete someone's database. >Turns out the ufw firewall I enabled and diligently kept on a strict allowlist with only my internal servers didn’t work on a new server because of Docker. When I containerized MongoDB, Docker helpfully inserted an allow rule into iptables, opening up MongoDB to the world. So while my firewall was “active”, doing a sudo iptables -L | grep 27017 showed that MongoDB was open the world. This has been a Docker footgun since 2014. Story was previously discussed on HN[1]. Sure, you could argue the author should have done more to secure the endpoint, but this was 100% a failure mode due to how Docker prioritizes convenience over security. [0] https://blog.newsblur.com/2021/06/28/story-of-a-hacking/ https://blog.newsblur.com/2021/06/28/story-of-a-hacking/ [1] https://news.ycombinator.com/item?id=27670058 https://news.ycombinator.com/item?id=27670058
      - pella 3y ago
- fouc 3y ago
  I wouldn't say "designed from the start", I would just say virtual machines are naturally more isolated than containers. Since virtual machines are simulating hardware, compared to containers which are isolated user space instances. But I guess that doesn't mean virtual machines aren't easily escapable without extra work, same as containers.
- ec109685 3y ago
  Linux was designed as a multi-user operating system from the get go. By definition, programs from different users are supposed to run without access to data that they shouldn’t have access to. That said, it’s hard to get right at all times.

lmm 3y ago

Because virtual machines were designed from the start to run untrusted code, and containers were not? > supposed to keep whatever is inside trapped unless you poke holes in that protection As far as I know that was never a design decision for containers on Linux; certainly not in the early days.

zekrioca 3y ago
This doesn’t answer the question. How were VMs designed with security in mind and not with host emulation? Untrusted code? I’m confused, people are talking about vulnerabilities.
- icedchai 3y ago
  VMs don’t share the kernel with the host. Any host escape would need to happen through a device driver exposed to the guest (virtio, etc.) Containers use the same kernel, obviously a much larger set of code. More code means a greater chance of vulnerabilities.
  - zekrioca 3y ago
    How do you think the VM itself is spawned? Something in the host instantiate it. But even if we ignore that, I would argue that concentrating in one location only (e.g., some exposed driver) to escape is also easier, since an attacker would need to spend time trying to find only one vulnerability rather than several. I think the hardware can help bridging the gap between containers and VMs by enabling userspace processes behave as VMs, which is more or less what QEMU+KVM try to do, except that it still comes with some overheads and less flexibility.
    - icedchai 3y ago
      I have to disagree. With a VM, there is less shared code shared with the host to review and audit for vulnerabilities. The developers can go over those device drivers, system calls for VM management, etc. with a fined-tooth comb. With containers, you have essentially the entire kernel, much more surface area to potentially exploit. Maybe I am wrong. We can wait for a security professional to comment.
- dharmab 3y ago
  When you switch between VMs and the host the CPU executes an instruction to isolate the data of each VMs entirely (flushing caches). This doesn't happen with containers.
  - zekrioca 3y ago
    Although this feature does improve security, it is due to the CPU, the VM and the main idea was to reduce performance overheads. I do not see adding layers and layers of abstractions the same as more secure. In fact, meltdown and spectre serve as counter-examples. Also, one can achieve similar effects with containers as well, just think AppArmor, capabilities, permissions, etc., all layers of administrative privileges between some untrusted code and the host.
    - dharmab 3y ago
      There is nothing you can do in the OS that replaces the VMENTER instruction. You need this precisely because it mitigates SPECTRE and Meltdown, assuming your microcode is up to date.
yjftsjthsd-h 3y ago
I'm willing to believe that Linux containers were not initially designed to be a security boundary, but I struggle to see why that means they aren't now; it's been over a decade and they have an awful lot of security features for something that doesn't care about security. EDIT: For that matter, they're clearly being used for security; the features in Linux that are used by runc et al. are the same features used by eg. Chrome to isolate components in order to contain vulnerabilities.
- fbdab103 3y ago
  As far as I know, Docker still punches through your firewall by default. I consider that a pretty big negative against assuming secure-by-design.
  - yjftsjthsd-h 3y ago
    What attack are you envisioning that is aided by docker bypassing firewall rules?
    - fbdab103 3y ago
      I mean, there was this story[0] ("How a Docker footgun led to a vandal deleting NewsBlur's MongoDB database") about how the Docker rules allowed a hacker to delete someone's database. >Turns out the ufw firewall I enabled and diligently kept on a strict allowlist with only my internal servers didn’t work on a new server because of Docker. When I containerized MongoDB, Docker helpfully inserted an allow rule into iptables, opening up MongoDB to the world. So while my firewall was “active”, doing a sudo iptables -L | grep 27017 showed that MongoDB was open the world. This has been a Docker footgun since 2014. Story was previously discussed on HN[1]. Sure, you could argue the author should have done more to secure the endpoint, but this was 100% a failure mode due to how Docker prioritizes convenience over security. [0] https://blog.newsblur.com/2021/06/28/story-of-a-hacking/ https://blog.newsblur.com/2021/06/28/story-of-a-hacking/ [1] https://news.ycombinator.com/item?id=27670058 https://news.ycombinator.com/item?id=27670058
    - pella 3y ago
fouc 3y ago
I wouldn't say "designed from the start", I would just say virtual machines are naturally more isolated than containers. Since virtual machines are simulating hardware, compared to containers which are isolated user space instances. But I guess that doesn't mean virtual machines aren't easily escapable without extra work, same as containers.
ec109685 3y ago
Linux was designed as a multi-user operating system from the get go. By definition, programs from different users are supposed to run without access to data that they shouldn’t have access to. That said, it’s hard to get right at all times.