4 ms·

Integrate that with kubernetes and I'm sold.

by mrAssHat 2y ago

_joel 2y ago
What, like https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/ ?
- d3m0t3p 2y ago
  I think it's from the AWS team, they made firecracker (micro VM) So it does exist. Funnily that's what fly does: take your container uncompress it to a full micro VM and run it on their infra
  - deleted 2y ago
    [deleted]
  - iamstan23 2y ago
    fly.io uses Firecracker. Firecracker is Open Sourced with an Apache 2 license. It's faster than LightVM mentioned in the post. Firecracker also has containerd support (https://github.com/firecracker-microvm/firecracker-containerd https://github.com/firecracker-microvm/firecracker-container...). There are a few ways to run Kubernetes with Firecracker, including FireKube.
    - throwaway482945 2y ago
      Is it really faster? I thought firecracker boot times were something like 100ms. LightVM claims 2.3ms?
      - tpetry 2y ago
        LightVM is stating a VM creation of 2.3ms while Firecracker states 125ms of time from VM creation to a working user space. So this comparing apples and oranges.
        fhuici 2y ago
        The 125ms is using Linux. Using a unikernel and tweaking Firecracker a bit (on KraftCloud) we can get, for example, 20 millis cold starts for NGINX, and have features on the way to reduce this further.
        imtringued 2y ago
        I know it's cool to talk about these insane numbers, but from what I can tell people have AWS lambdas that boot slower than this to the point where people send warmup calls just to be sure. What exactly warrants the ability to start a VM this quickly?
      - fhuici 2y ago
        Back when we did the paper, Firecracker wasn't mainstream so we ended up doing a (much hackier) version of a fast VMM by modifying's Xen's VMM; but yeah, a few millis was totally feasible back then, and still now (the evolution of that paper is Unikraft, a LF OSS project at www.unikraft.org). (Cold) boot times are determined by a chain of components, including (1) the controller (eg, k8s/Borg), (2) the VMM (Firecracker, QEMU, Cloud Hypervisor), (3) the VM's OS (e.g., Linux, Windows, etc), (4) any initialization of processes, libs, etc and finally (5) the app itself. With Unikraft we build extremely specialized VMs (unikernels) in order to minimize the overhead of (3) and (4). On KraftCloud, which leverages Unikraft/unikernels, we additionally use a custom controller to optimize (1) and Firecracker to optimize (2). What's left is (5), the app, which hopefully the developers can optimize if needed.
crabbone 2y ago
There are NVidia's Kata containers: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-operator-kata.html https://docs.nvidia.com/datacenter/cloud-native/gpu-operator... . I'm not sure you need the physical GPUs to run them though. Most likely not. I'm wondering though what value will Kubernetes add beside integrating with existing (presumably Kubernetes-based) infrastructure? At least, this is my understanding of the rationale for Kata containers. Other than that, it seems like it'd be just getting in the way...
- bjconlan 2y ago
  I believe this work originated at Intel as "clear containers" (which I believe started life from an acquisition (but could be mixing this up...my memory isn't what it used to be). Either way it's great they are being used like this and at Nvidia (I know Alibaba cloud also use this tech)
  - fhuici 2y ago
    Yes, Kata started as clear containers. And yes, the main purpose is compatibility with containers -- though generally speaking, adding layers to the cloud stack never helps to make a deployment more efficient. On kraft.cloud we use Dockerfiles to specify app/filesystem, but then at deploy time automatically and transparently convert that to a specialized VM/unikernel for best performance.
- basemi 2y ago
  Se also: https://katacontainers.io/ https://katacontainers.io/
nderjung 2y ago
We ended up doing this over at https://unikraft.io https://unikraft.io :-)

Integrate that with kubernetes and I'm sold.

_joel 2y ago

What, like https://firecracker-microvm.github.io/ https://firecracker-microvm.github.io/ ?

d3m0t3p 2y ago
I think it's from the AWS team, they made firecracker (micro VM) So it does exist. Funnily that's what fly does: take your container uncompress it to a full micro VM and run it on their infra
- deleted 2y ago
  [deleted]
- iamstan23 2y ago
  fly.io uses Firecracker. Firecracker is Open Sourced with an Apache 2 license. It's faster than LightVM mentioned in the post. Firecracker also has containerd support (https://github.com/firecracker-microvm/firecracker-containerd https://github.com/firecracker-microvm/firecracker-container...). There are a few ways to run Kubernetes with Firecracker, including FireKube.
  - throwaway482945 2y ago
    Is it really faster? I thought firecracker boot times were something like 100ms. LightVM claims 2.3ms?
    - tpetry 2y ago
      LightVM is stating a VM creation of 2.3ms while Firecracker states 125ms of time from VM creation to a working user space. So this comparing apples and oranges.
      - fhuici 2y ago
        The 125ms is using Linux. Using a unikernel and tweaking Firecracker a bit (on KraftCloud) we can get, for example, 20 millis cold starts for NGINX, and have features on the way to reduce this further.
      - imtringued 2y ago
        I know it's cool to talk about these insane numbers, but from what I can tell people have AWS lambdas that boot slower than this to the point where people send warmup calls just to be sure. What exactly warrants the ability to start a VM this quickly?
    - fhuici 2y ago
      Back when we did the paper, Firecracker wasn't mainstream so we ended up doing a (much hackier) version of a fast VMM by modifying's Xen's VMM; but yeah, a few millis was totally feasible back then, and still now (the evolution of that paper is Unikraft, a LF OSS project at www.unikraft.org). (Cold) boot times are determined by a chain of components, including (1) the controller (eg, k8s/Borg), (2) the VMM (Firecracker, QEMU, Cloud Hypervisor), (3) the VM's OS (e.g., Linux, Windows, etc), (4) any initialization of processes, libs, etc and finally (5) the app itself. With Unikraft we build extremely specialized VMs (unikernels) in order to minimize the overhead of (3) and (4). On KraftCloud, which leverages Unikraft/unikernels, we additionally use a custom controller to optimize (1) and Firecracker to optimize (2). What's left is (5), the app, which hopefully the developers can optimize if needed.

crabbone 2y ago

There are NVidia's Kata containers: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-operator-kata.html https://docs.nvidia.com/datacenter/cloud-native/gpu-operator... . I'm not sure you need the physical GPUs to run them though. Most likely not. I'm wondering though what value will Kubernetes add beside integrating with existing (presumably Kubernetes-based) infrastructure? At least, this is my understanding of the rationale for Kata containers. Other than that, it seems like it'd be just getting in the way...

bjconlan 2y ago
I believe this work originated at Intel as "clear containers" (which I believe started life from an acquisition (but could be mixing this up...my memory isn't what it used to be). Either way it's great they are being used like this and at Nvidia (I know Alibaba cloud also use this tech)
- fhuici 2y ago
  Yes, Kata started as clear containers. And yes, the main purpose is compatibility with containers -- though generally speaking, adding layers to the cloud stack never helps to make a deployment more efficient. On kraft.cloud we use Dockerfiles to specify app/filesystem, but then at deploy time automatically and transparently convert that to a specialized VM/unikernel for best performance.
basemi 2y ago
Se also: https://katacontainers.io/ https://katacontainers.io/

nderjung 2y ago

We ended up doing this over at https://unikraft.io https://unikraft.io :-)