4 ms·

Deploying a hypervisor on top of a hypervisor (eg: a public cloud) usually makes very little sense so I don't see that as a trend. However, outside of the valle

by eyberg 7y ago

Deploying a hypervisor on top of a hypervisor (eg: a public cloud) usually makes very little sense so I don't see that as a trend. However, outside of the valley bubble many large sectors of the economy do not use public clouds (such as banking, health, defense) so hypervisors typically in the form of ESX is still very much a thing and now that we are seeing tighter integration with things like unikernels that is where the 'comeback' is coming from. I would almost go as far to say that it's becoming hip not to be using public cloud today. Deploying a unikernel on top of commodity cloud is very achievable and easy today - I know because I'm involved with https://ops.city https://ops.city . I'd argue it's ridiculously easier than trying to deploy a container, clearly way safer and even faster and easier than using something like terraform. I do very much agree with your impedance mismatch re: unix model thought though.

mynegation 7y ago
> However, outside of the valley bubble many large sectors of the economy do not use public clouds (such as banking, health, defense) I work in traditional finance, outside of Silicon Valley and there is a huge drive to move to the public cloud. For the same reason banks do not own their office buildings anymore - managing computing infrastructure is not their core business and - to be honest - they are not very good at it. I know people who have to move servers and applications to a different data centre because the lease on an old DC is ending, it is a huge headache, and they would much rather concentrate on their core business.
- eyberg 7y ago
  I don't know who you specifically work for but I've been to 2 different banking tech conferences in the past month. JPMorgan mentioned they have 13 (or was it 16?) wholly owned datacenters. BoA's tech summit was just this past week in Palo Alto and they were highly negative of anything public cloud. For them it is an extremely rare thing for them to put anything in public cloud. A point they stressed throughout the summit is that the main thing they sell is 'trust' (eg: by holding people's money) and that is quite simply not something they are willing to delegate to others and that's the main driver for them but of course all the other things like cost, security, regulatory, and data gravity are drivers as well.
  - Terretta 7y ago
    Since you cite JPMC, read this Sept 2019 interview with CEO Jamie Dimon: https://www.geekwire.com/2019/interview-jpmorgan-chase-ceo-jamie-dimon-ai-cloud-banks-big-bet-seattle-tech/ https://www.geekwire.com/2019/interview-jpmorgan-chase-ceo-j... “Jamie Dimon: The cloud and AI are real, huge, powerful and they’re going to change a lot for us. Of course, good companies are going to adapt to it in due course. They can’t just do it overnight, but it changes a lot more than just using the cloud — how you write your code, how you can do agile, it even changes how you organize your company. Managers start to think about using data in a far different way than they ever did before. It changes everything. It’s a huge opportunity.” ... “When cloud first got started, there was no reason to use someone else’s data center or their networks. You know, I wasn’t completely wrong, if I can run my data center as efficiently as a cloud guy, which fundamentally is true.” ”But here’s what I missed: the bursting part of the cloud. It reduces the peak capacity that we’ve got to keep in our own data centers, and you can burst excess workloads into the cloud. And it directly relates to AI. There’s just so much compute power in one split second that I simply can’t do in one of my data centers. So if you want to apply AI and these tools, and also the services that are built into some of these platforms [the cloud offers an advantage.]” // Disclaimer: I work at a global bank.
sansnomme 7y ago
Hypervisor on hypervisor makes sense if you care about security. Your average FaaS-on-Kubernetes hosting services not from one of the tech giants is a far cry from AWS Firecracker (Lambda) level of isolation. E.g. you run a gaming server and wants to allow arbitrary user plugins, or perhaps a challenger fintech bank that wants some simple user scripting. You want to allow Python/Ruby/any lang except Lua/JS/WASM. You won't be able to find anything hosted that doesn't cost an arm and a leg. (On an unrelated note, if anyone from Zeit is reading this, can you comment on why now.sh container support was deprecated in V2? Was it because of cost?)
- eyberg 7y ago
  I totally agree that if you care about security you should be running in the opposite direction of k8s. Google Cloud has this option (nested hypervisors) but not for security reasons - it's used to run other software that comes packaged as a vm yet that comes with a serious perf hit and which is why I don't see that ever catching on as a trend. Selling bare metal servers w/pre-installed hosted firecracker would make much more sense (if you want/need hosting). If you are interested in isolating a particular application to a given interpreter or a given binary the aforementioned unikernels are precisely what you are looking for and those can be provisioned on t2.micros.
  - sansnomme 7y ago
    Your analysis is on point and brilliant. I would upvote this more if I could. Not enough people on HN understand the weaknesses of Kubernetes and existing self-hosted cloud software. (And more importantly, what hypervisors brings to the table)
  - rcoveson 7y ago
    > I totally agree that if you care about security you should be running in the opposite direction of k8s. The parts of k8s that are sticky and hard to change are the API abstractions like Services, Deployments, and Nodes. That's also where a lot of the value add is. Like AWS cloud formation. The only part that is docker-specific are the docker images. I don't think the sandboxing techniques (i.e. docker) will stay the same for long. There are already multiple initiatives to implement hypervisor-based pods. I think firecracker has a lot of inertia right now, and I'm pretty sure RKT containers with KVM isolation were in beta a couple year ago. The point is, I don't see how kubernetes has all that much to do with security (at least as far as isolation goes). Docker alternatives will gain traction and k8s will offer them as pod runtimes. Maybe the direction you mean we should be running is the opposite one from Docker?
hamiltont 7y ago
Wanted to share my first experience with the ops.city site. This is unsolicited, but I do hope it helps your team succeed: I opened the 'Why' tab and found blurbs I wanted to see more about. For example, security through smaller code+attack surface sounds interesting, 'no ops' (didn't grok the bullets here, but I was interested to know more), and higher performance are all interesting The bullets were interesting enough for me to want more, so I opened the video. Unfortunately, the video was really vanilla and didn't expand on these concepts at all. It shows how easy ops.city is, but gives me nothing more on why it is better than ten other technologies that have similar 'easy intro' videos. What I was hoping for was a 5-10 minute video focused on benefits of your approach vs alternatives. For example, some talk or demo on improved performance, data on how much code/attack surface is reduced, comments on OS/kernel memory footprint (this is a problem for us - clients like using 500MB "tiny" VMs on public clouds that hang running cadvisor), etc. I could leave your site to answer these questions on my own. I did this and found some promising looking links over on the nanovm website. But that is an awkward experience - leaving your site to try and understand your offering. Perhaps I'm not your target audience. For context, I do a minor amount of devops as part of delivering initial R&D products to multiple clients. We are not ops experts, but we try to deliver future proof work, which involves us staying apprised of new approaches and making them available to clients when they are a fit.

by eyberg 7y ago

mynegation 7y ago

> However, outside of the valley bubble many large sectors of the economy do not use public clouds (such as banking, health, defense) I work in traditional finance, outside of Silicon Valley and there is a huge drive to move to the public cloud. For the same reason banks do not own their office buildings anymore - managing computing infrastructure is not their core business and - to be honest - they are not very good at it. I know people who have to move servers and applications to a different data centre because the lease on an old DC is ending, it is a huge headache, and they would much rather concentrate on their core business.

eyberg 7y ago
I don't know who you specifically work for but I've been to 2 different banking tech conferences in the past month. JPMorgan mentioned they have 13 (or was it 16?) wholly owned datacenters. BoA's tech summit was just this past week in Palo Alto and they were highly negative of anything public cloud. For them it is an extremely rare thing for them to put anything in public cloud. A point they stressed throughout the summit is that the main thing they sell is 'trust' (eg: by holding people's money) and that is quite simply not something they are willing to delegate to others and that's the main driver for them but of course all the other things like cost, security, regulatory, and data gravity are drivers as well.
- Terretta 7y ago
  Since you cite JPMC, read this Sept 2019 interview with CEO Jamie Dimon: https://www.geekwire.com/2019/interview-jpmorgan-chase-ceo-jamie-dimon-ai-cloud-banks-big-bet-seattle-tech/ https://www.geekwire.com/2019/interview-jpmorgan-chase-ceo-j... “Jamie Dimon: The cloud and AI are real, huge, powerful and they’re going to change a lot for us. Of course, good companies are going to adapt to it in due course. They can’t just do it overnight, but it changes a lot more than just using the cloud — how you write your code, how you can do agile, it even changes how you organize your company. Managers start to think about using data in a far different way than they ever did before. It changes everything. It’s a huge opportunity.” ... “When cloud first got started, there was no reason to use someone else’s data center or their networks. You know, I wasn’t completely wrong, if I can run my data center as efficiently as a cloud guy, which fundamentally is true.” ”But here’s what I missed: the bursting part of the cloud. It reduces the peak capacity that we’ve got to keep in our own data centers, and you can burst excess workloads into the cloud. And it directly relates to AI. There’s just so much compute power in one split second that I simply can’t do in one of my data centers. So if you want to apply AI and these tools, and also the services that are built into some of these platforms [the cloud offers an advantage.]” // Disclaimer: I work at a global bank.

sansnomme 7y ago

Hypervisor on hypervisor makes sense if you care about security. Your average FaaS-on-Kubernetes hosting services not from one of the tech giants is a far cry from AWS Firecracker (Lambda) level of isolation. E.g. you run a gaming server and wants to allow arbitrary user plugins, or perhaps a challenger fintech bank that wants some simple user scripting. You want to allow Python/Ruby/any lang except Lua/JS/WASM. You won't be able to find anything hosted that doesn't cost an arm and a leg. (On an unrelated note, if anyone from Zeit is reading this, can you comment on why now.sh container support was deprecated in V2? Was it because of cost?)

eyberg 7y ago
I totally agree that if you care about security you should be running in the opposite direction of k8s. Google Cloud has this option (nested hypervisors) but not for security reasons - it's used to run other software that comes packaged as a vm yet that comes with a serious perf hit and which is why I don't see that ever catching on as a trend. Selling bare metal servers w/pre-installed hosted firecracker would make much more sense (if you want/need hosting). If you are interested in isolating a particular application to a given interpreter or a given binary the aforementioned unikernels are precisely what you are looking for and those can be provisioned on t2.micros.
- sansnomme 7y ago
  Your analysis is on point and brilliant. I would upvote this more if I could. Not enough people on HN understand the weaknesses of Kubernetes and existing self-hosted cloud software. (And more importantly, what hypervisors brings to the table)
- rcoveson 7y ago
  > I totally agree that if you care about security you should be running in the opposite direction of k8s. The parts of k8s that are sticky and hard to change are the API abstractions like Services, Deployments, and Nodes. That's also where a lot of the value add is. Like AWS cloud formation. The only part that is docker-specific are the docker images. I don't think the sandboxing techniques (i.e. docker) will stay the same for long. There are already multiple initiatives to implement hypervisor-based pods. I think firecracker has a lot of inertia right now, and I'm pretty sure RKT containers with KVM isolation were in beta a couple year ago. The point is, I don't see how kubernetes has all that much to do with security (at least as far as isolation goes). Docker alternatives will gain traction and k8s will offer them as pod runtimes. Maybe the direction you mean we should be running is the opposite one from Docker?

hamiltont 7y ago

Wanted to share my first experience with the ops.city site. This is unsolicited, but I do hope it helps your team succeed: I opened the 'Why' tab and found blurbs I wanted to see more about. For example, security through smaller code+attack surface sounds interesting, 'no ops' (didn't grok the bullets here, but I was interested to know more), and higher performance are all interesting The bullets were interesting enough for me to want more, so I opened the video. Unfortunately, the video was really vanilla and didn't expand on these concepts at all. It shows how easy ops.city is, but gives me nothing more on why it is better than ten other technologies that have similar 'easy intro' videos. What I was hoping for was a 5-10 minute video focused on benefits of your approach vs alternatives. For example, some talk or demo on improved performance, data on how much code/attack surface is reduced, comments on OS/kernel memory footprint (this is a problem for us - clients like using 500MB "tiny" VMs on public clouds that hang running cadvisor), etc. I could leave your site to answer these questions on my own. I did this and found some promising looking links over on the nanovm website. But that is an awkward experience - leaving your site to try and understand your offering. Perhaps I'm not your target audience. For context, I do a minor amount of devops as part of delivering initial R&D products to multiple clients. We are not ops experts, but we try to deliver future proof work, which involves us staying apprised of new approaches and making them available to clients when they are a fit.