5 ms·
Firecracker: Lightweight Virtualization for Serverless Applications (2020)
- isabellat 5y agoReally excited about this and rust-vmm which allows you to build custom Virtual Machine Monitors. The problem with a solution like Firecracker, and why they are able to get such fast startup times and a small memory footprint is that they selected a subset of features to expose. This limits the capabilities of the running container, which is of course the point. But it is not general purpose enough for all workloads. For instance, I want to use Firecracker but require host file system sharing. Rust-vmm looks like it is trying to solve this problem by providing a collection of rust crates that allow users build their own VMM with the features they need. It's build-a-bear for VMMs :)
- w7 5y agoYep! A project I use with Kubernetes is CloudHypervisor[0] (with kata-containers[1]). This is a rust-vmm based vmm that has VirtioFS (host FS sharing) support while still being leaner than QEMU. Very neat stuff. [0] https://github.com/cloud-hypervisor/cloud-hypervisor https://github.com/cloud-hypervisor/cloud-hypervisor [1] https://github.com/kata-containers/kata-containers https://github.com/kata-containers/kata-containers
- mjb 5y ago> It's build-a-bear for VMMs :) Yeah! Nearly everybody says they want a simple and minimal solution in this space, but everybody means different things when they say that. For example, in Lambda we do everything at the block level, and intentionally don't want to share filesystems (one reason is that it exposes the host kernel's complex FS code to the guest). But you want filesystems, and that's cool. Instead of having a box that does everything, with the associated size and attack surface, rust-vmm moves that feature binding to build time. It's slightly less convenient, but much more powerful. (I'm the guy in the video on the linked page, but this is just my opinion, nothing official)
- wey-gu 5y agoUsed to contribute to rust-vmm years before ;)
- Dunedan 5y agoDidn't get rust-vmm started way after Firecracker to avoid duplications in Firecracker and CrosVM?
- exikyut 5y agoIt's a niche use-case, but one of the things I've been curious about is nested virtualization using Firecracker as the outside VMM and KVM as the inner VMM (with QEMU being the sole userspace process running in the Firecracker VM). This would make it possible to use Firecracker for whole-system virtualization, and run Windows. KVM-in-Firecracker is apparently technically possible and just works thanks to Linux' support of nested virtualization, but celebratory noises to that effect elicited a slightly freaked out response: https://github.com/firecracker-microvm/firecracker/issues/1721 https://github.com/firecracker-microvm/firecracker/issues/17... That issue only really covered the security impliciations of multiple KVM (QEMU) instances inside a given Firecracker instance though, which I can totally see as (strictly, pedantically) problematic. But if QEMU is literally the only thing running (besides an only-what's-necessary /init) in a given Firecracker instance, that sounds to me like I'd have the best of all the worlds (Firecracker's awesome security and hardware emulation) with verifiably no downsides, right? The only ramifications I can see are if the hardware acceleration involved in nested virtualization might represent a security vulnerability. Theoretically a practical attack would equivalently impact Firecracker-inside-Firecracker use cases as well, on the assumption that it would need to first break out of QEMU and then Firecracker, as opposed to *waves hands* doing something that punches all the way from the twice-VMM'd guest all the way out to the host because of a hardware fault. Such hardware issues are arguably unfixable though (insert standard paragraph about microcode updates here), so theoretically (from a software boundary correctness perspective) I should be good to go right?
- apitman 5y agoFirecracker is absolutely awesome. Looking at the dismal state of sandboxing APIs in the major operating systems, and the advent of KVM on Linux, WHP on Windows (including Windows Home!), and HVF on Mac, I wonder if the future of securely running applications might be to just throw every app in a VM, similar to how Qubes OS works. Technologies like Looking Glass[0] can even enable 3D acceleration. While Firecracker is faster, QEMU does actually have a microvm[0] machine which boots pretty dang quickly. QEMU also has the advantage of user space networking which allows guests to access the internet without any admin privileges required. Unfortunately the network speeds are fairly slow. I'm hopeful that this can be made much faster moving forward. QEMU also works on Windows, which is huge. But I haven't been able to get microvms to work on Windows hosts yet. [0]: https://looking-glass.io/ https://looking-glass.io/ [1]: https://qemu.readthedocs.io/en/latest/system/i386/microvm.html https://qemu.readthedocs.io/en/latest/system/i386/microvm.ht...
- NovemberWhiskey 5y ago>I wonder if the future of securely running applications might be to just throw every app in a VM If you're using AWS Lambda, or ECS Fargate, you're already there, right?
- anderspitman 5y agoSorry, I was ambiguous. I'm talking about desktop applications.
- pjmlp 5y agoThe future, catching up the past introduced in mainframes. Everything old is new again.
- anderspitman 5y agoThere are similarities, sure. But unless I'm misunderstanding what you mean, running a desktop application in a fully 3D-accelerated VM that's completely sandboxed from the rest of your system isn't anything that's been widely done before that I know of.
- torvald 5y agoThis is also what fly.io is using underneath their Nomad orchestrator. https://fly.io/docs/about/open-source/ https://fly.io/docs/about/open-source/
- kamilafsar 5y agoAFAIK Amazon as a company is not really open sourcing a lot of their software. I was surprised that they would open source such a key component of their Lambda offering. As someone here already mentioned, it certainly helped Fly.io steeling some of their customers. (Although Fly has no serverless offering, but uses it for a docker-ish deployment model.) And last time I checked there were a few more startups using it. Can anyone (maybe Amazon folks here) shed a light on the rationale behind it, why they’d open sourced it and what they could possibly gain by it?
- anderspitman 5y agoI'm not sure, but maybe because it started as a fork of crosvm[0]? [0]: https://github.com/google/crosvm https://github.com/google/crosvm