3 ms·

From a technical perspective, Amazon has actually optimized this but turned that into "serverless functions": their ultra-optimized image paired with Firecracke

From a technical perspective, Amazon has actually optimized this but turned that into "serverless functions": their ultra-optimized image paired with Firecracker achieve ultra-fast boot-up of virtual Linux machines. IIRC from when Firecracker was being introduced, they are booting up in sub-second times. I wonder if Amazon would ever decide to offer booting the same image with the same hypervisor in EC2 as they do for lambdas?

arianvanp 2y ago
And AWS now has a product to spin up Lambdas for GitHub Actions CI runners https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html https://docs.aws.amazon.com/codebuild/latest/userguide/actio...
cr125rider 2y ago
Fargate is an alternative that runs on Firecracker as well. It's hidden behind ECS and EKS, however.
- scarface_74 2y ago
  And CodeBuild…
- mochomocha 2y ago
  According to [1] Fargate is actually not using Firecracker, but probably something closer to a single container running in a single-tenant ec2 VM. If true, this makes VM boot-time optimizations and warm pooling even more important for such product. [1]: https://justingarrison.com/blog/2024-02-08-fargate-is-not-firecracker/ https://justingarrison.com/blog/2024-02-08-fargate-is-not-fi...
- plopz 2y ago
  Fargate is too slow without the container cache you can get with ec2
20thr 2y ago
100% -- EC2's general purpose nature is not in my opinion the best fit for ephemeral use-cases. You'll be constantly fighting the infrastructure as the set of trade-offs and design goals are widely different. This is why CodeSandbox, Namespace, and even fly.io built special-purpose architectures to guarantee extremely start-up time. In the case of Namespace it's ~2sec on cold boots with a set of user-supplied containers, with storage allocations. (Disclaimer, I'm with Namespace -- https://namespace.so https://namespace.so)
fhuici 2y ago
[Disclaimer: I'm with KraftCloud] For what it's worth, Firecracker/the VMM is only one part of the boot process. Among others, there's also the controller and the VM/OS itself that typically slow things down. In other words, it's not enough to just switch in Firecracker and expect cold starts to immediately drop to sub-second levels. On kraft.cloud we've fundamentally redesigned the cloud stack to be able to cold start containers/VMs in milliseconds (eg, about 20 millis for nginx, about 50 millis for a basic Node app), and also scale them to zero and autoscale them in milliseconds. If interested there's more info about the tech in our blog posts https://unikraft.io/blog/ https://unikraft.io/blog/ .
solarengineer 2y ago
Not an AWS example, but on my Illumos Zones on an i5 at Hetzner, I get from zero to ssh in under 50 ms. I am certain of the numbers since I have used DTrace to measure. It is unfortunate that Ilumos is not popular enough for a multitude of reasons.
- solarengineer 2y ago
  I should add - this is a full Zone with a number of services. One of my half-finished projects was to trace the call graph of the various services started and disable those not needed.
- necovek 2y ago
  Wow, that's amazing! I wonder if I should try out Illumos as I am rebuilding my home server, but I am afraid that due to lack of time, it'd take me ages to replicate my libvirt-based setup with around 30 services. How well are Linux containers and VMs supported in Illumos? How about nested KVM? That could help with the transition as I am heavily leaning into GNU tools and KVM. FWIW, when I first laid my eyes on a Sun workstation back in the 90s and seen the Control key where it rightly belongs (in the place of Caps Lock in broken layouts :), I said "duh" and have never moved back from remapped Caps-as-Ctrl.
  - solarengineer 2y ago
    You could use SmartOS to achieve the same. Linux containers no longer run natively and instead need to run within VMs.
- oblio 2y ago
  > It is unfortunate that Ilumos is not popular enough for a multitude of reasons. It's complicated but you can blame Sun for a lot of that, the primary thing was their explicit choice of license to make it incompatible (or hard-to-make-compatible with Linux).

arianvanp 2y ago

And AWS now has a product to spin up Lambdas for GitHub Actions CI runners https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html https://docs.aws.amazon.com/codebuild/latest/userguide/actio...

cr125rider 2y ago

Fargate is an alternative that runs on Firecracker as well. It's hidden behind ECS and EKS, however.

scarface_74 2y ago
And CodeBuild…
mochomocha 2y ago
According to [1] Fargate is actually not using Firecracker, but probably something closer to a single container running in a single-tenant ec2 VM. If true, this makes VM boot-time optimizations and warm pooling even more important for such product. [1]: https://justingarrison.com/blog/2024-02-08-fargate-is-not-firecracker/ https://justingarrison.com/blog/2024-02-08-fargate-is-not-fi...
plopz 2y ago
Fargate is too slow without the container cache you can get with ec2

20thr 2y ago

100% -- EC2's general purpose nature is not in my opinion the best fit for ephemeral use-cases. You'll be constantly fighting the infrastructure as the set of trade-offs and design goals are widely different. This is why CodeSandbox, Namespace, and even fly.io built special-purpose architectures to guarantee extremely start-up time. In the case of Namespace it's ~2sec on cold boots with a set of user-supplied containers, with storage allocations. (Disclaimer, I'm with Namespace -- https://namespace.so https://namespace.so)

fhuici 2y ago

[Disclaimer: I'm with KraftCloud] For what it's worth, Firecracker/the VMM is only one part of the boot process. Among others, there's also the controller and the VM/OS itself that typically slow things down. In other words, it's not enough to just switch in Firecracker and expect cold starts to immediately drop to sub-second levels. On kraft.cloud we've fundamentally redesigned the cloud stack to be able to cold start containers/VMs in milliseconds (eg, about 20 millis for nginx, about 50 millis for a basic Node app), and also scale them to zero and autoscale them in milliseconds. If interested there's more info about the tech in our blog posts https://unikraft.io/blog/ https://unikraft.io/blog/ .

solarengineer 2y ago

Not an AWS example, but on my Illumos Zones on an i5 at Hetzner, I get from zero to ssh in under 50 ms. I am certain of the numbers since I have used DTrace to measure. It is unfortunate that Ilumos is not popular enough for a multitude of reasons.

solarengineer 2y ago
I should add - this is a full Zone with a number of services. One of my half-finished projects was to trace the call graph of the various services started and disable those not needed.
necovek 2y ago
Wow, that's amazing! I wonder if I should try out Illumos as I am rebuilding my home server, but I am afraid that due to lack of time, it'd take me ages to replicate my libvirt-based setup with around 30 services. How well are Linux containers and VMs supported in Illumos? How about nested KVM? That could help with the transition as I am heavily leaning into GNU tools and KVM. FWIW, when I first laid my eyes on a Sun workstation back in the 90s and seen the Control key where it rightly belongs (in the place of Caps Lock in broken layouts :), I said "duh" and have never moved back from remapped Caps-as-Ctrl.
- solarengineer 2y ago
  You could use SmartOS to achieve the same. Linux containers no longer run natively and instead need to run within VMs.
oblio 2y ago
> It is unfortunate that Ilumos is not popular enough for a multitude of reasons. It's complicated but you can blame Sun for a lot of that, the primary thing was their explicit choice of license to make it incompatible (or hard-to-make-compatible with Linux).