4 ms·

Hypervisor on hypervisor makes sense if you care about security. Your average FaaS-on-Kubernetes hosting services not from one of the tech giants is a far cry f

Hypervisor on hypervisor makes sense if you care about security. Your average FaaS-on-Kubernetes hosting services not from one of the tech giants is a far cry from AWS Firecracker (Lambda) level of isolation. E.g. you run a gaming server and wants to allow arbitrary user plugins, or perhaps a challenger fintech bank that wants some simple user scripting. You want to allow Python/Ruby/any lang except Lua/JS/WASM. You won't be able to find anything hosted that doesn't cost an arm and a leg. (On an unrelated note, if anyone from Zeit is reading this, can you comment on why now.sh container support was deprecated in V2? Was it because of cost?)

eyberg 7y ago
I totally agree that if you care about security you should be running in the opposite direction of k8s. Google Cloud has this option (nested hypervisors) but not for security reasons - it's used to run other software that comes packaged as a vm yet that comes with a serious perf hit and which is why I don't see that ever catching on as a trend. Selling bare metal servers w/pre-installed hosted firecracker would make much more sense (if you want/need hosting). If you are interested in isolating a particular application to a given interpreter or a given binary the aforementioned unikernels are precisely what you are looking for and those can be provisioned on t2.micros.
- sansnomme 7y ago
  Your analysis is on point and brilliant. I would upvote this more if I could. Not enough people on HN understand the weaknesses of Kubernetes and existing self-hosted cloud software. (And more importantly, what hypervisors brings to the table)
- rcoveson 7y ago
  > I totally agree that if you care about security you should be running in the opposite direction of k8s. The parts of k8s that are sticky and hard to change are the API abstractions like Services, Deployments, and Nodes. That's also where a lot of the value add is. Like AWS cloud formation. The only part that is docker-specific are the docker images. I don't think the sandboxing techniques (i.e. docker) will stay the same for long. There are already multiple initiatives to implement hypervisor-based pods. I think firecracker has a lot of inertia right now, and I'm pretty sure RKT containers with KVM isolation were in beta a couple year ago. The point is, I don't see how kubernetes has all that much to do with security (at least as far as isolation goes). Docker alternatives will gain traction and k8s will offer them as pod runtimes. Maybe the direction you mean we should be running is the opposite one from Docker?

eyberg 7y ago

I totally agree that if you care about security you should be running in the opposite direction of k8s. Google Cloud has this option (nested hypervisors) but not for security reasons - it's used to run other software that comes packaged as a vm yet that comes with a serious perf hit and which is why I don't see that ever catching on as a trend. Selling bare metal servers w/pre-installed hosted firecracker would make much more sense (if you want/need hosting). If you are interested in isolating a particular application to a given interpreter or a given binary the aforementioned unikernels are precisely what you are looking for and those can be provisioned on t2.micros.

sansnomme 7y ago
Your analysis is on point and brilliant. I would upvote this more if I could. Not enough people on HN understand the weaknesses of Kubernetes and existing self-hosted cloud software. (And more importantly, what hypervisors brings to the table)
rcoveson 7y ago
> I totally agree that if you care about security you should be running in the opposite direction of k8s. The parts of k8s that are sticky and hard to change are the API abstractions like Services, Deployments, and Nodes. That's also where a lot of the value add is. Like AWS cloud formation. The only part that is docker-specific are the docker images. I don't think the sandboxing techniques (i.e. docker) will stay the same for long. There are already multiple initiatives to implement hypervisor-based pods. I think firecracker has a lot of inertia right now, and I'm pretty sure RKT containers with KVM isolation were in beta a couple year ago. The point is, I don't see how kubernetes has all that much to do with security (at least as far as isolation goes). Docker alternatives will gain traction and k8s will offer them as pod runtimes. Maybe the direction you mean we should be running is the opposite one from Docker?