4 ms·

It's still containers. On a cloud provider the Kubernetes workers are VM's which orchestrate containers. With Kata Containers you're just spawning containers in

It's still containers. On a cloud provider the Kubernetes workers are VM's which orchestrate containers. With Kata Containers you're just spawning containers inside micro-vm's.

int0x80 8y ago
That is one big difference, with big implications.
- stevenacreman 8y ago
  Just wanted to clarify that Kubernetes was still scheduling containers. Even if VM's are being used to isolate them. It's not all or nothing either. Containerd will support running a mix of containers and kata-containers across workers. For anyone interested in this topic I wrote about some other container runtimes here: https://kubedex.com/kubernetes-container-runtimes/ https://kubedex.com/kubernetes-container-runtimes/
  - amscanne 8y ago
    Note that this is not the case for virtual kubelet-based implementations, and your point here and above are specific to how Kata works (the article is talking more generally).
  - int0x80 8y ago
    Yeah.. I think I see what you say, I mean, the end user interface is the quite the same but this has big implications anyway for the systems design POV, so is not a small thing :)
jacques_chester 8y ago
> It's still containers. The security profiles of containers and VMs, including kernel-based VMs, are different. VMs still have a significant edge, because the attack surface is smaller and doesn't have many competing missions.
- tptacek 8y ago
  The attack surface argument is debatable, depending on how the system is designers, since virtualization introduces the hypervisor surface.
  - jacques_chester 8y ago
    The hypervisor surface can be made smaller, since its major goal is to manage hardware resources. A kernel has the same mission, but also has a mission to provide a rich API for applications.
  - halbritt 8y ago
    The attack surface argument certainly is debatable. I wonder how many multi-tenant workloads are actually at risk of an escape vulnerability. I wager that the multi-tenancy described in the article in the OP is actually disparate workloads across disparate teams in a particular enterprise where it seems (to me) fairly unlikely for someone with access to run a workload to also have the willingness to compile and run malicious code to take advantage of an escape vulnerability. On the other hand, publicly available compute, i.e. AWS, GCP, Azure seems way more likely to be the subject of attacks from random malicious individuals seek to take advantage of an escape vulnerability if one existed.
- cyphar 8y ago
  The attack surface of a container can be massively reduced with seccomp profiles -- there was a paper a few years ago which found that the effective attack surface of a hypervisor was about the same as the attack surface of a locked-down seccomp profile of a container (and LXC/Docker/etc already have a default whitelist profile which has in practice mitigated something like 90% of kernel 0days). And let's not forget the recent CPU exploits which found that VMs aren't very separated after all. The fact that Kubernetes disables this (and other) security features by default should be seen as a flaw in Kubernetes. (Just as some of the flaws of Docker should be seen as Docker flaws not containers-in-general flaws.)
  - ryacko 8y ago

It's still containers. On a cloud provider the Kubernetes workers are VM's which orchestrate containers. With Kata Containers you're just spawning containers inside micro-vm's.

int0x80 8y ago

That is one big difference, with big implications.

stevenacreman 8y ago
Just wanted to clarify that Kubernetes was still scheduling containers. Even if VM's are being used to isolate them. It's not all or nothing either. Containerd will support running a mix of containers and kata-containers across workers. For anyone interested in this topic I wrote about some other container runtimes here: https://kubedex.com/kubernetes-container-runtimes/ https://kubedex.com/kubernetes-container-runtimes/
- amscanne 8y ago
  Note that this is not the case for virtual kubelet-based implementations, and your point here and above are specific to how Kata works (the article is talking more generally).
- int0x80 8y ago
  Yeah.. I think I see what you say, I mean, the end user interface is the quite the same but this has big implications anyway for the systems design POV, so is not a small thing :)

jacques_chester 8y ago

> It's still containers. The security profiles of containers and VMs, including kernel-based VMs, are different. VMs still have a significant edge, because the attack surface is smaller and doesn't have many competing missions.

tptacek 8y ago
The attack surface argument is debatable, depending on how the system is designers, since virtualization introduces the hypervisor surface.
- jacques_chester 8y ago
  The hypervisor surface can be made smaller, since its major goal is to manage hardware resources. A kernel has the same mission, but also has a mission to provide a rich API for applications.
- halbritt 8y ago
  The attack surface argument certainly is debatable. I wonder how many multi-tenant workloads are actually at risk of an escape vulnerability. I wager that the multi-tenancy described in the article in the OP is actually disparate workloads across disparate teams in a particular enterprise where it seems (to me) fairly unlikely for someone with access to run a workload to also have the willingness to compile and run malicious code to take advantage of an escape vulnerability. On the other hand, publicly available compute, i.e. AWS, GCP, Azure seems way more likely to be the subject of attacks from random malicious individuals seek to take advantage of an escape vulnerability if one existed.
cyphar 8y ago
The attack surface of a container can be massively reduced with seccomp profiles -- there was a paper a few years ago which found that the effective attack surface of a hypervisor was about the same as the attack surface of a locked-down seccomp profile of a container (and LXC/Docker/etc already have a default whitelist profile which has in practice mitigated something like 90% of kernel 0days). And let's not forget the recent CPU exploits which found that VMs aren't very separated after all. The fact that Kubernetes disables this (and other) security features by default should be seen as a flaw in Kubernetes. (Just as some of the flaws of Docker should be seen as Docker flaws not containers-in-general flaws.)
- ryacko 8y ago