4 ms·
I think the “Firecracker” referenced in this Tweet is Amazon's “microVM” environment[0]. VMs running in this environment are sometimes booted to service schedul
by MrDOS 3y ago
I think the “Firecracker” referenced in this Tweet is Amazon's “microVM” environment[0]. VMs running in this environment are sometimes booted to service scheduled/batch jobs, but are, I think, more often launched in direct response to (and with the intent of serving) an incoming HTTP request (and then stick around for a few minutes afterwards in case there are any subsequent requests). Network latency from the continental US to the nearest AWS datacentre is typically <30ms, so even after you add the response time of the application running in the VM, an extra 2ms boot time still probably makes up >2% of the total cold response time.
[0]: https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/
- trasz4 3y agoThis will also make a lot of difference once we can run lightweight, userspace kernels ("rump kernels").
- yjftsjthsd-h 3y ago> once we can run lightweight, userspace kernels ("rump kernels"). Er, didn't NetBSD accomplish that... like a decade ago, now? Or is FreeBSD looking into supporting rump kernels?
- trasz4 3y agoYes, NetBSD did accomplish that, but I expect it to become more commonplace.
- EuropeOverlords 3y agobut question is, is this solution for people running software on other peoples computers ? or solution for running my own code on my own computer only ?
- yjftsjthsd-h 3y agoDoes it matter? Running kernel pieces in userspace is useful in both directions (rather like docker), and in both cases we'd like it to run as quickly as possible.
- trasz4 3y agoFor my particular case it's about something similar in purpose to sandboxing, but with providing the compartment (ie a process subtree) with an alternative kernel to talk to, to minimise the attack surface between that container and the host kernel.
- chongli 3y agoSo wouldn't this fall under the category of "booting the kernel too much"? Why not find a way to keep already-booted kernels laying around and recycle them, rather than rebooting them afresh on every request? This pattern has already played out in other situations and been solved this way, such as with object pools or thread pools.
- NovemberWhiskey 3y agoThis is the "AWS Lambda cold start" use case: you're not doing this every time, but you are doing it on scale-up, initial deploy, etc. Depending on complexity, those costs start to hit your p99 as well as your p100.
- deleted 3y ago[deleted]
- pavon 3y agoAll that sounds like it adds much more complexity than swapping out bubble sort for qsort, and the other various optimizations that have been going in to reduce boot time. Plus optimizing common code improves boot time for everyone running FreeBSD - routers, firewalls, etc, whereas pooling just helps that one use case. If you can make it startup up fast enough that you don't need any of that extra machinery, why wouldn't you?
- insanitybit 3y agoYes, Firecracker supports snapshotting for that reason. But you still have the famous "cold start" lag. Not to mention that, for some use cases, you'll potentially want to add restarts for security reasons (Firecracker is designed to be able to execute malicious code).
- nerpderp82 3y agoThe problem here isn't the boot time, but the booting at all!!
- giantrobot 3y agoHaving a microVM handle a rare endpoint and then exit is often cheaper than leaving an EC2 instance running constantly. An EC2 instance also has extra administration needs. At some scales the savings can be significant. Shaving ms off the boot of that VM is a performance boost of that lambda (or whatever the cloud vendor calls it).
- nerpderp82 3y agoWhy does anything have to boot? What does booting do?
- chatmasta 3y agoMaybe you're calling the serverless function from another serverless function in the same datacenter (or more simply, maybe you're instantiating a local firecracker VM to run some user-defined function within the context of your currently executing program), in which case the latency would be more on the order of 5-10ms.