4 ms·

Which defeats the purpose of containerization because now the container can sniff traffic on the host, hijack TCP connections, etc.

stingraycharles 6y ago
I know many people use single tenant hardware to run containers, so I wouldn’t say that it defeats the purpose of containerization, just one of the benefits.
- jeffbee 6y ago
  The confusion and disagreement on this topic is rooted in faulty assumptions about other organizations' requirements. Many people use containers with host networking, without uid namespaces, without pid namespaces, without bind mounts. What you think is "a container" may not be at all universal. Also the idea that host networking grants all processes CAP_NET_RAW is just weird and wrong.
MaxBarraclough 6y ago
If you're running untrusted code in your container, you've pretty much already lost. Containers are useful for deployment and configuration, they are not a robustly secure sandbox. For that you still need to go with a VM. No cloud provider will offer to run your containers alongside other customer's containers, on a shared kernel. Your containers always run within your own VM.
- jeffbee 6y ago
  I have to say, it would be cool if they did. If I could get a dirt-cheap rate for running batch workloads in a potentially antagonistic environment, I could make use of that. Not all data is sensitive.
  - MaxBarraclough 6y ago
    It wouldn't save you that much compared to just going with a VM, especially if it's computationally intensive.
  - cpuguy83 6y ago
    It is how Heroku has operated since forever. Containers can be secure.
    - MaxBarraclough 6y ago
      I didn't know that about Heroku. They give you a choice: > Performance and Private dynos do not share the underlying compute instance with other dynos https://devcenter.heroku.com/articles/dynos#isolation-and-security https://devcenter.heroku.com/articles/dynos#isolation-and-se...
- effie 6y ago
  No, it is levels of security, not a white/black issue. For some customers and providers, containered processes are enough isolation. For many small businesses, even shared hosting with chroot is enough. Generally speaking, Xen or Firecracker VMs do have smaller attack surface than containered processes on a shared Linux kernel. But configuration and exposed capabilities matter - it is possible to have container better secured than a VM (e.g. minimal Zones/jails env + correct MAC config vs. general Qemu/VMware VM with many default legacy devices and bad/no MAC config). Motivated attackers can escape even these VMs. So they are not a magical solution. Common hypervisors are too big and buggy to be pronounced as security panacea. From time to time, VM escapes resurface to public but most are probably guarded and being exploited in quiet. As we know after Spectre and Meltdown, standard computing technology is buggy/bugged all the way down to hardware. If you want really "robustly secure" server environment, such do exist: for example, separation kernels like the L4 family or the Green Hills INTEGRITY systems. But for web apps, almost nobody bothers. > No cloud provider will offer to run your containers alongside other customer's containers, on a shared kernel. Your containers always run within your own VM. Joyent does - via SmartOS zones.
  - MaxBarraclough 6y ago
    > From time to time, VM escapes resurface to public but most are probably guarded and being exploited in quiet. As we know after Spectre and Meltdown, standard computing technology is buggy/bugged all the way down to hardware. To my knowledge there has never been a successful escape from the VMs offered by AWS, GCP, or Azure. That would be a pretty big story. > If you want really "robustly secure" server environment, such do exist: for example, separation kernels like the L4 family or the Green Hills INTEGRITY systems. But for web apps, almost nobody bothers. What's the reason none of the major cloud providers use seL4? Missing features? By seL4's own account their performance is exceptional, but perhaps its performance can't compete against a hardware-assisted system like AWS Nitro? > Joyent does - via SmartOS zones. Thanks I'd not heard of that.
    - effie 6y ago
      You won't hear much about the VM escapes in those clouds because there is strong incentive for almost everyone to be quiet about them and patch as soon as possible so there "isn't a story". Some researchers do want to make a name for themselves and publish before the patch, but those are minority. And publicly attacking a big corporation is quite risky so they prefer presenting the bug as general problem in the technology. But the bugs in hypervisors exist, see CVE's for Xen for the past decade. See also https://security.stackexchange.com/questions/130274/how-do-big-cloud-providers-guard-against-vm-escape-attacks https://security.stackexchange.com/questions/130274/how-do-b... https://nakedsecurity.sophos.com/2015/05/14/the-venom-virtual-machine-escape-bug-what-you-need-to-know/ https://nakedsecurity.sophos.com/2015/05/14/the-venom-virtua... Regarding L4, I do not know. Probably it is very different and cumbersome to work with compared to linux.
m463 6y ago
I believe the purpose of containers is different for different people. build vs deploy.

stingraycharles 6y ago

I know many people use single tenant hardware to run containers, so I wouldn’t say that it defeats the purpose of containerization, just one of the benefits.

jeffbee 6y ago
The confusion and disagreement on this topic is rooted in faulty assumptions about other organizations' requirements. Many people use containers with host networking, without uid namespaces, without pid namespaces, without bind mounts. What you think is "a container" may not be at all universal. Also the idea that host networking grants all processes CAP_NET_RAW is just weird and wrong.

MaxBarraclough 6y ago

If you're running untrusted code in your container, you've pretty much already lost. Containers are useful for deployment and configuration, they are not a robustly secure sandbox. For that you still need to go with a VM. No cloud provider will offer to run your containers alongside other customer's containers, on a shared kernel. Your containers always run within your own VM.

jeffbee 6y ago
I have to say, it would be cool if they did. If I could get a dirt-cheap rate for running batch workloads in a potentially antagonistic environment, I could make use of that. Not all data is sensitive.
- MaxBarraclough 6y ago
  It wouldn't save you that much compared to just going with a VM, especially if it's computationally intensive.
- cpuguy83 6y ago
  It is how Heroku has operated since forever. Containers can be secure.
  - MaxBarraclough 6y ago
    I didn't know that about Heroku. They give you a choice: > Performance and Private dynos do not share the underlying compute instance with other dynos https://devcenter.heroku.com/articles/dynos#isolation-and-security https://devcenter.heroku.com/articles/dynos#isolation-and-se...
effie 6y ago
No, it is levels of security, not a white/black issue. For some customers and providers, containered processes are enough isolation. For many small businesses, even shared hosting with chroot is enough. Generally speaking, Xen or Firecracker VMs do have smaller attack surface than containered processes on a shared Linux kernel. But configuration and exposed capabilities matter - it is possible to have container better secured than a VM (e.g. minimal Zones/jails env + correct MAC config vs. general Qemu/VMware VM with many default legacy devices and bad/no MAC config). Motivated attackers can escape even these VMs. So they are not a magical solution. Common hypervisors are too big and buggy to be pronounced as security panacea. From time to time, VM escapes resurface to public but most are probably guarded and being exploited in quiet. As we know after Spectre and Meltdown, standard computing technology is buggy/bugged all the way down to hardware. If you want really "robustly secure" server environment, such do exist: for example, separation kernels like the L4 family or the Green Hills INTEGRITY systems. But for web apps, almost nobody bothers. > No cloud provider will offer to run your containers alongside other customer's containers, on a shared kernel. Your containers always run within your own VM. Joyent does - via SmartOS zones.
- MaxBarraclough 6y ago
  > From time to time, VM escapes resurface to public but most are probably guarded and being exploited in quiet. As we know after Spectre and Meltdown, standard computing technology is buggy/bugged all the way down to hardware. To my knowledge there has never been a successful escape from the VMs offered by AWS, GCP, or Azure. That would be a pretty big story. > If you want really "robustly secure" server environment, such do exist: for example, separation kernels like the L4 family or the Green Hills INTEGRITY systems. But for web apps, almost nobody bothers. What's the reason none of the major cloud providers use seL4? Missing features? By seL4's own account their performance is exceptional, but perhaps its performance can't compete against a hardware-assisted system like AWS Nitro? > Joyent does - via SmartOS zones. Thanks I'd not heard of that.
  - effie 6y ago
    You won't hear much about the VM escapes in those clouds because there is strong incentive for almost everyone to be quiet about them and patch as soon as possible so there "isn't a story". Some researchers do want to make a name for themselves and publish before the patch, but those are minority. And publicly attacking a big corporation is quite risky so they prefer presenting the bug as general problem in the technology. But the bugs in hypervisors exist, see CVE's for Xen for the past decade. See also https://security.stackexchange.com/questions/130274/how-do-big-cloud-providers-guard-against-vm-escape-attacks https://security.stackexchange.com/questions/130274/how-do-b... https://nakedsecurity.sophos.com/2015/05/14/the-venom-virtual-machine-escape-bug-what-you-need-to-know/ https://nakedsecurity.sophos.com/2015/05/14/the-venom-virtua... Regarding L4, I do not know. Probably it is very different and cumbersome to work with compared to linux.

m463 6y ago

I believe the purpose of containers is different for different people. build vs deploy.