3 ms·

There was a question (now deleted) on difference between this and Firecracker. FC/gVisor/Kata use VMs to sandbox containers. However, VMs incur high runtime ove

There was a question (now deleted) on difference between this and Firecracker. FC/gVisor/Kata use VMs to sandbox containers. However, VMs incur high runtime overhead (not to be confused with fast booting) and need to be provisioned. We use hardware virtualization to directly run system/app containers -- no I/O emulation, no expensive VM exits, scale as needed.

rad_gruchalski 5y ago
I was the one who posted that question and I deleted it because I understood I was asking to compare apple to oranges. Firecracker uses KVM under the hood. With regards to the overhead, sure, there's going to be resources needed but firecracker is pretty good at running VMs will low level overhead. We're talking thousands on a single host. I've done some fair share of evaluating firecracker for https://github.com/combust-labs/firebuild https://github.com/combust-labs/firebuild and the need to provision is red herring. firebuild can run a VM directly from a Dockerfile and Docker image. Fly.io team does something similar. It's basically a fully functional app out of the container within a matter of milliseconds. What would be nice is to see a direct comparison between your solution and firecracker.
- tptacek 5y ago
  (I'm a developer at Fly). I hesitated to chime in with specifics because I haven't done any real work on timing and profiling and because my dev environment is a NUC which, while zippy, is not the state of the art execution environment for Firecracker. But, long story short: I restart test Firecrackers constantly, from shell scripts, and from start to my Go application logging its first "I'm listening" message after its VM has fully initialized, it's essentially instantaneous. I'm sure it could be made faster, but making it faster would in no meaningful way change my life. Instances on Fly can take a second or two to come up, but that's orchestration overhead, not Firecracker (Fly is zippy, too, don't get me wrong.)
  - deleted 5y ago
    [deleted]
  - rad_gruchalski 5y ago
    That’s definitely true for no orchestration. If the kernel and rootfs is at the jailer location, things are instant. You guys have your owninit system which makes things pretty fast, indeed. I rely on whatever init system is the default for a given os so that adds an overhead because it implies running the app as an upstart / systemd / whatever service. Love your work by the way.
new_realist 5y ago
So, like gVisor's KVM support?

rad_gruchalski 5y ago

I was the one who posted that question and I deleted it because I understood I was asking to compare apple to oranges. Firecracker uses KVM under the hood. With regards to the overhead, sure, there's going to be resources needed but firecracker is pretty good at running VMs will low level overhead. We're talking thousands on a single host. I've done some fair share of evaluating firecracker for https://github.com/combust-labs/firebuild https://github.com/combust-labs/firebuild and the need to provision is red herring. firebuild can run a VM directly from a Dockerfile and Docker image. Fly.io team does something similar. It's basically a fully functional app out of the container within a matter of milliseconds. What would be nice is to see a direct comparison between your solution and firecracker.

tptacek 5y ago
(I'm a developer at Fly). I hesitated to chime in with specifics because I haven't done any real work on timing and profiling and because my dev environment is a NUC which, while zippy, is not the state of the art execution environment for Firecracker. But, long story short: I restart test Firecrackers constantly, from shell scripts, and from start to my Go application logging its first "I'm listening" message after its VM has fully initialized, it's essentially instantaneous. I'm sure it could be made faster, but making it faster would in no meaningful way change my life. Instances on Fly can take a second or two to come up, but that's orchestration overhead, not Firecracker (Fly is zippy, too, don't get me wrong.)
- deleted 5y ago
  [deleted]
- rad_gruchalski 5y ago
  That’s definitely true for no orchestration. If the kernel and rootfs is at the jailer location, things are instant. You guys have your owninit system which makes things pretty fast, indeed. I rely on whatever init system is the default for a given os so that adds an overhead because it implies running the app as an upstart / systemd / whatever service. Love your work by the way.

new_realist 5y ago

So, like gVisor's KVM support?