4 ms·

A Study of Firecracker and gVisor (2020) [pdf]

44 points by eatonphil 5y ago

lima 5y ago
Note that has been a number of recent performance improvements in gVisor (VFS2, soon LisaFS). Here's a recent blog post about performance in a large production setup: https://gvisor.dev/blog/2021/12/02/running-gvisor-in-production-at-scale-in-ant https://gvisor.dev/blog/2021/12/02/running-gvisor-in-product...
vlovich123 5y ago
I don't know if number of kernel lines executed is a good baseline for establishing the exposure of a given sandboxing approach.
- vayne 5y ago
  Also, the number of lines of kernel code being executed is not a valid metric without comparing how much of those code flow can the user control. In gVisor, this is mostly limited to what the go runtime can call. There is very little direct control over these syscalls.
mjb 5y ago
> Narrowing the interface of a platform will not necessarily improve security, unless it reduces the footprint of host code likely to contain vulnerabilities, so we have focused on understanding the kernel footprint underneath each virtualization system. These things are hard to quantify, but this is a very reasonable attempt. However, as we say in one of our Firecracker papers (https://www.usenix.org/conference/nsdi20/presentation/agache https://www.usenix.org/conference/nsdi20/presentation/agache): > Li et al [32] demonstrate the effectiveness of a ’popular paths’ metric, showing that only 3% of kernel bugs are found in frequently-used code paths (which, in our experience, overlap highly with the code paths used by the VMM) There, the reference is to Lock-In-Pop by Li et al (https://www.usenix.org/conference/atc17/technical-sessions/presentation/li-yiwen https://www.usenix.org/conference/atc17/technical-sessions/p...). This is important, because bug density isn't constant across the kernel surface area (just as it isn't constant across any code base). FWIW I don't think Lock-In-Pop is the right mechanism for isolation, but it's a very good way to weight the metrics if we're comparing kernel surface area. Back to the paper at hand: > Both gVisor and Firecracker restrict the set of system calls available to improve security and isolation. I'm not sure what they mean here. Firecracker doesn't restrict the set of system calls guests can use. In my mind, that's one of the biggest advantages of Firecracker's approach - there's no hard tradeoff to be made between isolation strength and guest software compatibility.
- amscanne 5y ago
  > I'm not sure what they mean here. Firecracker doesn't restrict the set of system calls guests can use. In my mind, that's one of the biggest advantages of Firecracker's approach - there's no hard tradeoff to be made between isolation strength and guest software compatibility. gVisor is also not artificially restricting this set (it is limited by what's implemented, but that's not a hard trade-off); the sentence means that both Firecracker and gVisor restrict the host surface available to the effective "supervisor", either the VMM or the Sentry.
  - vayne 5y ago
    +1. I'd also add that gVisor's slightly limited compatibility is a security feature. Like there is no need for obscure functionality which most production apps rarely use.
unixhero 5y ago
Are firecracker and gvisor available as containers in Ubuntu / Debian?
- unixhero 5y ago
  Follow-up question, can they be implemented in Proxmox? How feasible is it.
jason2323 5y ago
Tyler is an absolutely fantastic instructor and mentor! Glad to see his work show up on the front page!

44 points by eatonphil 5y ago

lima 5y ago

Note that has been a number of recent performance improvements in gVisor (VFS2, soon LisaFS). Here's a recent blog post about performance in a large production setup: https://gvisor.dev/blog/2021/12/02/running-gvisor-in-production-at-scale-in-ant https://gvisor.dev/blog/2021/12/02/running-gvisor-in-product...

vlovich123 5y ago

I don't know if number of kernel lines executed is a good baseline for establishing the exposure of a given sandboxing approach.

vayne 5y ago
Also, the number of lines of kernel code being executed is not a valid metric without comparing how much of those code flow can the user control. In gVisor, this is mostly limited to what the go runtime can call. There is very little direct control over these syscalls.

mjb 5y ago

> Narrowing the interface of a platform will not necessarily improve security, unless it reduces the footprint of host code likely to contain vulnerabilities, so we have focused on understanding the kernel footprint underneath each virtualization system. These things are hard to quantify, but this is a very reasonable attempt. However, as we say in one of our Firecracker papers (https://www.usenix.org/conference/nsdi20/presentation/agache https://www.usenix.org/conference/nsdi20/presentation/agache): > Li et al [32] demonstrate the effectiveness of a ’popular paths’ metric, showing that only 3% of kernel bugs are found in frequently-used code paths (which, in our experience, overlap highly with the code paths used by the VMM) There, the reference is to Lock-In-Pop by Li et al (https://www.usenix.org/conference/atc17/technical-sessions/presentation/li-yiwen https://www.usenix.org/conference/atc17/technical-sessions/p...). This is important, because bug density isn't constant across the kernel surface area (just as it isn't constant across any code base). FWIW I don't think Lock-In-Pop is the right mechanism for isolation, but it's a very good way to weight the metrics if we're comparing kernel surface area. Back to the paper at hand: > Both gVisor and Firecracker restrict the set of system calls available to improve security and isolation. I'm not sure what they mean here. Firecracker doesn't restrict the set of system calls guests can use. In my mind, that's one of the biggest advantages of Firecracker's approach - there's no hard tradeoff to be made between isolation strength and guest software compatibility.

amscanne 5y ago
> I'm not sure what they mean here. Firecracker doesn't restrict the set of system calls guests can use. In my mind, that's one of the biggest advantages of Firecracker's approach - there's no hard tradeoff to be made between isolation strength and guest software compatibility. gVisor is also not artificially restricting this set (it is limited by what's implemented, but that's not a hard trade-off); the sentence means that both Firecracker and gVisor restrict the host surface available to the effective "supervisor", either the VMM or the Sentry.
- vayne 5y ago
  +1. I'd also add that gVisor's slightly limited compatibility is a security feature. Like there is no need for obscure functionality which most production apps rarely use.

unixhero 5y ago

Are firecracker and gvisor available as containers in Ubuntu / Debian?

unixhero 5y ago
Follow-up question, can they be implemented in Proxmox? How feasible is it.

jason2323 5y ago

Tyler is an absolutely fantastic instructor and mentor! Glad to see his work show up on the front page!