8 ms·
AWS Adds support for nested virtualization
- igtztorrero 8mo agoDigital Ocean has always supported nested virtualization.
- sitole 8mo agoSupport for nested virtualization has been added to the main SDKs. In the us-west-2 region, you can already see the "Nested Virtualization" option and use it with the new M8id, C8id, and R8id instance types. This is really big news for micro-VM sandbox solutions like E2B, which I work on.
- blibble 8mo agowelcome AWS to 2018!
- ssl-3 8mo agoYep. It's pretty boring. I've been using it at home for years and years with libvirt on very not-special consumer hardware. I guess the AWS clown is finally catching up on this one little not-new-at-all thing.
- otterley 8mo agoI was an Amazon EC2 Specialist SA in a prior role, so I know a little about this. If EC2 were like your home server, you might be right. And an EC2 bare metal instance is the closest approximation to that. On bare metal, you've always been free to run your own VMs, and we had some customers who rolled their own nested VM implementations on it. But EC2 is not like your home server. There are some nontrivial considerations and requirements to offer nested virtualization at cloud scale: 1. Ensuring virtualized networking (VPC) works with nested VMs as well as with the primary VM 2. Making sure the environment (VMM etc) is sufficiently hardened to meet AWS's incredibly stringent security standards so that nesting doesn't pose unintended threats or weaken EC2's isolation properties. EC2 doesn't use libvirt or an off-the-shelf KVM. See https://youtu.be/cD1mNQ9YbeA?si=hcaZaV2W_hcEIn9L&t=1095 https://youtu.be/cD1mNQ9YbeA?si=hcaZaV2W_hcEIn9L&t=1095 and https://youtu.be/hqqKi3E-oG8?si=liAfollyupYicc_L&t=501 https://youtu.be/hqqKi3E-oG8?si=liAfollyupYicc_L&t=501 3. Ensuring performance and reliability meets customer standards 4. Building a rock-solid control plane around it all It's not a trivial matter of flipping a bit.
- QuinnyPig 8mo agoI always enjoy the color you add to these conversations. Thanks!
- sien 8mo agoI always enjoy the color you add to these conversations in your newsletter. It's provided many a chuckle. Thanks!
- sitole 8mo agoNitro is very interesting stuff
- raw_anon_1111 8mo agoSeriously curious, don’t Firecracker VMs already run on EC2 instances under the hood when they host Lambda and Fargate?
- otterley 8mo agoUnfortunately I'm not at liberty to dive deep into those details. I will say that Firecracker can be used on bare metal EC2 instances, whether you're a public customer or AWS itself. :-)
- raw_anon_1111 8mo agoI guess I should have peeked at the source code when I was there…
- rescbr 8mo agoNo need, at least when I was there when the day was still one, before the pandemic. And well, Firecracker is open source. A few of the best technical presentations that I've watched were at a pre-SKO event. Nitro, Graviton and Firecracker. Great engineering pieces, the three of them.
- 8mo ago
- tryauuum 8mo agothe only thing I know about nested virtualization is from the libvirt/KVM world too: * you are right, it just works * but there were scary notes about the stuff which might happen when you live migrate a virtual machine between hypervisors and the machine has nested virtual machines inside it. I remember the words "neither safe nor secure"
- yencabulator 8mo ago> * but there were scary notes about the stuff which might happen when you live migrate a virtual machine between hypervisors and the machine has nested virtual machines inside it. I remember the words "neither safe nor secure" Google does this to customer VMs in production all the time..
- deleted 8mo ago[deleted]
- gchamonlive 8mo agoHighly doubt that
- farklenotabot 8mo agoSounds expensive for legacy apps
- bagels 8mo ago"* *Feature*: Launching nested virtualization. This feature allows you to run nested VMs inside virtual (non-bare metal) EC2 instances."
- ATechGuy 8mo agoWould love to see performance numbers with nested virtualization, particularly that of IO-bound workloads.
- api 8mo agoWhat's the performance impact for nested virtualization in general? I'd think this would be adding multiple layers of MMU overhead.
- dwattttt 8mo agoFrom memory, the virtualisation operations themselves aren't nested. The VM instructions interact with the external virtualisation hardware, so it's more of a cooperative situation, e.g. a guest can create & manage virtualisation structures that are run alongside it. I don't know if this applies to the specific nested virtualisation AWS are providing though.
- otterley 8mo agoAs a practical matter, anywhere from 5-15%.
- blibble 8mo agodepends on the workload and how they've done it pure CPU should be essentially unaffected, if they're not emulating the MMU/page tables in software the difference in IO ranges from barely measurable to absolutely horrible, depending on their implementation traps/vmexits have another layer to pass through (and back)
- gerdesj 8mo agoCould someone explain why this is might be a big deal? I remember playing with nested virty some years ago and deciding it is a backwards step except for PoC and the like. Given I haven't personally run out of virty gear, I never needed to do a PoC.
- paulfurtado 8mo agoIt is great for isolation. There are so many VM based containerization solutions at this point, like Kata Containers, gvisor, and Firecracker. With kata, your kubernetes pods run in isolated VMs. It also opens the door for live migration of apps between ec2 instances, making some kinds of maintenance easier when you have persistent workloads. Even if not for security, there are so many ways a workload can break a machine such that you need to reboot or replace (like detaching an ebs volume with a mounted xfs filesystem at the wrong moment). The place I've probably wanted it the most though is in CI/CD systems: it's always been annoying to build and test system images in EC2 in a generic way. It also allows for running other third party appliances unmodified in EC2. But also, almost every other execution environment offers this: GCP, VMWare, KVM, etc, so it's frustrating that EC2 has only offered it on their bare metal instance types. When ec2 was using xen 10+ years ago, it made sense, but they've been on kvm since the inception of nitro.
- iangudger 8mo agoOne of the big benefits that gVisor offers is that it doesn't require nested virtualization (or any virtualization). They released a new version that improves performance when not using virtualization a while back: https://gvisor.dev/blog/2023/04/28/systrap-release/ https://gvisor.dev/blog/2023/04/28/systrap-release/
- UltraSane 8mo agoYou can now run VMs inside a cheaper AWS instance instead of having to pay for an entire bare-metal instance. This is useful for things like network simulation where you use QEMU to emulate network hardware.
- deleted 8mo ago[deleted]
- anurag 8mo agoThis is a big deal because you can now run Firecracker/other microVMs in an AWS VM instead of expensive AWS bare-metal instances. GCP has had nested virtualization for a while.
- parhamn 8mo agowhats the ~ perf hit of something like this?
- iJohnDoe 8mo agoWas hoping this comment would be here. Firecracker and microVMs are good use-case. Also, being able to simply test and develop is a nice to have. Nested virtualization can mean a lot of things. Not just full VMs.
- HumanOstrich 8mo ago> Firecracker and microVMs are good use-case. Good use-case for what?
- adobrawy 8mo agoNowadays universal answer for "what? why?" is AI. AI agent needs VMs to run generated code in sandbox as they can not be trusted.
- 8mo ago
- dangoodmanUT 8mo agohell yes, finally
- dk8996 8mo agoWould these thing be good for openclaw, agents?
- CuriouslyC 8mo agoYeah, though honestly if I'm deploying anything I'd just build an image with nix rather than use nested virtualization.
- boulos 8mo agoI feel vindicated :). We put in a lot of effort with great customers to get nested virtualization running well on GCE years ago, and I'm glad to hear AWS is coming around. You can tell people to just do something else, there's probably a separate natural solution, etc. but sometimes you're willing to sacrifice some peak performance just have that uniformity of operations and control.
- ilaksh 8mo agoI wonder if providers like Hetzner and Digital Ocean etc. will get this someday also.
- ubanholzer 8mo agoDO has Nested Virtualization enabled for years.
- ohthehugemanate 8mo agoI wonder if this is connected to Azure launching OpenShift Virtualization on "Boost" SKUs? There are a lot of VMWare customers going to OpenShift Virt, and apparently the CPU/memory overhead on Azure maxes out around 10% under full load... but then hyper V has been doing a lot of work on it. No idea if nitro includes any of the KVM-on-KVM passthrough of full KVM, to give it an edge here.
- wmf 8mo agoAzure? OpenShift? "I don't think about you at all." — Matt Garman probably
- _zoltan_ 8mo agoyou might not but a lot of very big enterprises use openshift on azure.
- firesteelrain 8mo agoAzure has had nested virt for a while - maybe it’s related to OpenShift but you could run OpenShift on Azure for some time. I used to run HyperV in Azure on certain SKUs
- zxspectrum1982 8mo agoOpenShift Virtualization on AWS, even as a managed service ("ROSA Virtualization"), has been available for a while on bare metal. Theoretically this enables ROSA Virtualization on EC2, in case you had valid reasons for such a thing.
- aliljet 8mo agoI wonder if this will extend SEV-SNP and TDX to the child VMs?
- leetrout 8mo agoIt says VSM is automatically disabled... so I would assume not.
- leetrout 8mo ago> Nested virtualization is supported only on 8th generation Intel-based instance types (c8i, m8i, r8i, and their flex variants). When nested virtualization is enabled, Virtual Secure Mode (VSM) is automatically disabled for the instance.
- BobbyTables2 8mo agoIs nested VMX virtualization in the Linux kernel really that stable? The technical details are a lot more complex than most realize. Single level VMX virtualization is relatively straightforward even if there are a lot of details to juggle with VMCS setup and handing exits. Nested virtualization is a whole another animal as one now also has to handle not just the levels but many things the hardware normally does, plus juggling internal state during transitions between levels. The LKML is filled with discussions and debates where very sharp contributors are trying to make sense of how it would work. Amazon turning the feature on is one thing. It working 100% perfectly is quite another…
- HumanOstrich 8mo agoIt's been around for almost 15 years and stable enough for several providers to roll it out in production the past 10 years (GCP and Azure in 2017). AWS is just late to the game because they've rolled so much of their own stack instead of adapting open source solutions and contributing back to them.
- otterley 8mo ago> AWS is just late to the game because they've rolled so much of their own stack instead of adapting open source solutions and contributing back to them. This is emphatically not true. Contributing to KVM and the kernel (which AWS does anyway) would not have accelerated the availability. EC2 is not just a data center with commodity equipment. They have customer demands for security and performance that far exceed what one can build with a pile of OSS, to the extent that they build their own compute and networking hardware. They even have CPU and other hardware SKUs not available to the general public.
- briffle 8mo agoAs do all the other cloud providers, that have had this for years. like GCP and Azure, for 9 years now.
- 8mo ago
- dostick 8mo agoProof that we’re living in a simulation.
- wmf 8mo agoI haven't watched The Thirteenth Floor in a while. The kids today don't even know about it.
- blaz0 8mo agoThis will make it easier to run automated tests in the Android emulator in CI using regular runners. It was a pain dealing with bare-metal instances just for that.
- andrewstuart 8mo agoOnly took them 9 years. AWS so much innovation. Remember, “customer obsession”. But “protect revenue first”.
- otterley 8mo agoThere’s no revenue protection here. You pay the same for an instance whether you’ve subdivided it into your own VMs or not.
- alexellisuk 8mo agoThis is great news for folks that use microVMs - "we only use AWS" has been an issue for our stuff (slicer services/sandboxes/actuated self-hosted GitHub runners) If anyone here can't wait (as it looks like there's very little info on this at the moment..) I wrote up detailed instructions for Ant Group's KVM-PVM patches. Performance is OK for background servers/tasks, but does take a hit up to 50% on complex builds like Kernels or Go with the K8s client. DIY/detailed option: https://blog.alexellis.io/how-to-run-firecracker-without-kvm-on-regular-cloud-vms/ https://blog.alexellis.io/how-to-run-firecracker-without-kvm... Fully working, pre-built host and guest kernel and rootfs: https://docs.slicervm.com/tasks/pvm/ https://docs.slicervm.com/tasks/pvm/ I'll definitely be testing this and comparing as soon as it's available. Hopefully it'll be accelerated somewhat compared to the PVM approach. There's still no sign whether those patches will ever end up merged upstream in the Linux Kernel. If you know differently, I'd appreciate a link. Azure, OCI, DigitalOcean, GCE all support nested virt as an option and do all take a bit of a hit, but it makes for very easy testing / exploration. Bare-metal on Hetzner now has a setup fee of up to 350 EUR.. you can find some stuff with 0 setup fee, but it's usually quite old kit. Edit: this doesn't look quite as good as the headline.. Options for instances look a bit limited. Someone found some more info here: https://x.com/nanovms/status/2022141660143165598/photo/1 https://x.com/nanovms/status/2022141660143165598/photo/1
- PunchyHamster 8mo ago> Bare-metal on Hetzner now has a setup fee of up to 350 EUR.. you can find some stuff with 0 setup fee, but it's usually quite old kit. I don't understand what you are paying for here, nested virtualization doesn't need any extra setup for hardware compared to normal one ... or you are saying Hetzner wants 350 EUR for turning on normal virtualization option in BIOS ?
- krab 8mo agoHetzner charges a fee for setting up your bare-metal machine. Often zero for their smaller machines and for those in auction. Probably they don't want someone to order a large fleet large of machines for one month and then cancel. They might not get another customer for those machines soon.
- anentropic 8mo agoIs this only when using the Go SDK?
- otterley 8mo agoNah, it’ll show up in the others in their upcoming releases. Much of the code for the SDKs is autogenerated from JSON “API shape” files: https://github.com/aws/api-models-aws https://github.com/aws/api-models-aws Specifically, in this case: https://github.com/aws/api-models-aws/commit/8bca88a33592ca4fb7faeec75b2afce057d1fbe8 https://github.com/aws/api-models-aws/commit/8bca88a33592ca4...
- amne 8mo agoobligatory: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... spoiler though: I'm referencing the part where gimp is running in Wine running in asm.js in a Chrome browser running in another asm.js in Firefox
- amelius 8mo agoBut I'm sure their ToS doesn't allow you to run your own cloud platform inside AWS.
- cthalupa 8mo agoWhy wouldn't it? Tons of their customers are providing cloud-like offerings hosted on AWS. They're getting paid either way.
- lofties 8mo agoYo dawg, I heard you like virtualisation so we put virtual servers inside of your virtual servers.
- j45 8mo agoIt also makes me wonder how many other things I might not know that people are trying to do with cloud platforms that aren’t supported by them but have a negligible performance hit for many use cases.
- csummers 8mo agoThis a great news, but is there any more information about this other than an aws sdk commit? Is this generally available?
- la64710 8mo agoHow can this be replicated on prem?
- fersarr 8mo agoWhen will AWS add a statement about being bound to professional secrecy (e.g s203 in Germany) so we use the LLM endpoints for sensitive industries https://repost.aws/es/questions/QUOuFPk9TLSUuClI_wYNmVCQ/serving-users-who-are-bound-by-professional-secrecy-doctors-lawyers-etc https://repost.aws/es/questions/QUOuFPk9TLSUuClI_wYNmVCQ/ser...
- windsor 8mo agoWow. Literally spent half a week planning a multi-quarter roadmap catering to working with bare metal. Half of that document is now deprecated LOL.