6 ms·

We clone a running VM in 2 seconds (2022)

169 points by hunvreus 2y ago

NitpickLawyer 2y ago
Cool article! The stack (and results) are impressive, but I also appreciate the article in itself, starting from basics and getting to the point in a clear and slowly expanding way. Easy to follow and appreciate. On a bit of a tangent rant, this kind of writing is slowly going away, taken over by LLM slop (and I'm a huge fan of LLMs, just not the people who write those kinds of articles). I was recently looking for real world benchmarks for vllm/sglang deployments of DeepSeek3 on a 8x 96GB pod, to see if the model fits into the amount of RAM, with kv cache and context length, what numbers to people get, etc. Of the ~20 articles that google surfaced on various attempts of keywords, none were what I was looking for. The excerpts seemed promising, some even offered tables & stuff related to ds3 and RAM usage, but all were LLM crap. All were written in that simple style - intro - bla bla - conclusion, some even had RAM requirements that made no sense (running a model trained in FP8 in 16bit, something noone would do, etc.)
- fxtentacle 2y ago
  While I fully agree with you on the absence of good benchmarks and the growing LLM slop ... "running a model trained in FP8 in 16bit, something noone would do, etc" I did that because on the RTX 3090 - which can be a good bang per buck for inference - the FP8 support is nerfed at the driver level. So a kernel that upscales FP8 to FP16 inside SRAM, then does the matmul, then downscales to FP8 again can bring massive performance benefits on those consumer cards. BTW, you can run a good DeepSeek3 quant on a single H200.
  - NitpickLawyer 2y ago
    Thanks! I was looking at blackwell 6000PROs, 8x 96GB for running full fp8 (as it's supported and presumably fast). I know AWQ should run, and be pretty snappy & efficient w/ the new MLA added, but wanted to check if fp8 fits as well, because from a simple napkin math it seems pretty tight (might only work for bs1, ctx_len <8k which would probably not be suited for coding tasks).
- thomasjudge 2y ago
  You are describing this as a writing problem, but it sounds more like a search results/search engine problem
1970-01-01 2y ago
The VM vs container debate is fascinating. They are separate yet slowly merging concepts that are becoming more blurred as technology becomes cheaper and faster. If the real bottleneck to scale is adaptable code, then it is foolish to dismiss the VM as outdated tech when it can be completely rehomed in 2 seconds. That megabyte of python code managing your containers would still be busy checking it's dependencies in that same timeframe.
- znpy 2y ago
  In a way it's nothing new. MOSIX/openMosix (https://en.wikipedia.org/wiki/MOSIX https://en.wikipedia.org/wiki/MOSIX, https://en.wikipedia.org/wiki/OpenMosix https://en.wikipedia.org/wiki/OpenMosix) did similar stuff with individual processes. It would probably be even faster as then you would have to only move process memory and its state rather than the whole VM memory (and it state). I guess it would/could be nice to have something that moves kubernetes pods around rather killing them and starting new ones.
- jiggawatts 2y ago
  IMHO the real advancement was the implicit snapshot Docker takes after every line in a Dockerfile. Virtual Machine builder scripts like Packer could do this… but don’t. It’s a choice, not an inherent technology limitation.
  - dontlaugh 2y ago
    I have the opposite opinion, the implicit overlay filesystem in Docker is an unnecessary and frustrating complication.
simonklitj 2y ago
Interesting read—thanks! One question: in the CoW example, if VM A modifies the data post-fork, what does VM B see when it later copies that data? Does it get the original data from the time of the fork, or VM A’s modified version?
- CompuIves 2y ago
  I talk a bit about this here: https://codesandbox.io/blog/cloning-microvms-using-userfaultfd https://codesandbox.io/blog/cloning-microvms-using-userfault.... Before VM A updates its data, the data is copied over to VM B if VM B hadn't written/read that data yet.
  - simonklitj 2y ago
    clever! Thank you.
londons_explore 2y ago
Unmentioned: there are serious security issues with memory cloning code not designed for it. For example, an SSL library might have pre-calculated the random nonce for the next incoming SSL connection. If you clone the VM containing a process using that library, now both child VM's will use the same nonce. Some crypto is 100% broken open if a nonce is reused.
- generalizations 2y ago
  Sounds like it would simply be inappropriate to clone & use a VM that's assuming it's data is unique. This would also be true of other conditions, e.g. if you needed to spoof a MAC or IPv6 address & picked one randomly.
  - londons_explore 2y ago
    The problem is modern software is so fiendishly complicated there almost certainly is stuff like that in the code. The question is where, and does it matter?
    - generalizations 2y ago
      And the last question is, can the parts with stuff like that be extracted from the rest and run separately?
- hypeatei 2y ago
  > might have pre-calculated the random nonce Isn't this still a concern even if you're not pre-calculating way ahead of time? If you generate it when needed, it could still catch you at the wrong time (e.g. right before encryption, but right after nonce generation)
  - zamadatix 2y ago
    Unless your encryption and transport protocols are 100% stateless only 1 connection will actually be able to form, even if you duplicate the machine during connection creation. The problem with pre-computing a bunch and keeping them in memory is brand new connections made post cloning would use the same list of nonces.
pragma_x 2y ago
I'm starting to see a pattern here. This describes a technology that rapidly deploys "VM" instances in the cloud which support things like Lambda and single-process containers. At what point do we scale this all back to a more rudimentary OS that provides security and process management across multiple physical machines? Or is there already a Linux distro that does this? I ask because watching cloud providers like AWS slowly reinvent mainframes just seems like the painful way around.
- no_wizard 2y ago
  EDIT: leaving the answer, but I simply misinterpreted what they meant. This isn't the same thing BSD has had jails for a long time, which let you achieve isolation on a system in this manner, or at least close to it.
  - hedora 2y ago
    They’re missing multi-machine orchestration: Run thousands of jails on these dozen machines. Don’t bother me with the details at runtime. They are also missing an ergonomic tool like dockerfiles. The following file, plus a cli tool for “run N copies on my M machines” should be enough to run bsd in prod, and it is not: “FROM openbsd:latest ; CMD pkg -i apache ; echo “apache=enabled >> /etc/rc.defaults ; COPY public_html /var/www/ ; CMD init” I don’t think writing the tooling would be that difficult, but it was missing the last time I looked.
    - no_wizard 2y ago
      I think I may have simply misinterpreted what you meant. You're right, its not Dockerfile-esque easy
- jerf 2y ago
  We've been cycling around that wheel for a while. If there's any difference now versus the past, it is that I think right now pretty much every point on the wheel is available quite readily now. If you want a more "rudimentary OS" you don't need to wait for the next turning of the wheel, it's here now. Need full VMs? Still a practical technology. Containers enough? Actively in development and use. Mix & match? Around any sensible combination you can do it now. And so on.
mystraline 2y ago
Different proposal: Let's say we have 2 Linux machines. Identical hardware, identical libs. I'd like to run a simple program on one machine, and then during mid-calculation, would like to transfer the running program to the other machine. Is this doable?
- new_user_final 2y ago
  Unrelated, but somewhat similar in higher level, you can transfer state with durable execution, e.g temporal.io. Instead of RAM, program's state saved in DB and execution environment resume in the previous state when restarted
  - WJW 2y ago
    How does such a method retain things like open network connections that have significant kernel state involved as well?
    - dilyevsky 2y ago
      it does not. all the state that you need to make "durable" needs to be explicitly committed in temporal via their sdk
- dilyevsky 2y ago
  Yes - using Criu[0] or docker checkpoint/restore api (which uses criu) [0] -https://criu.org/Main_Page https://criu.org/Main_Page
- panki27 2y ago
  Interesting thought, but highly dependant on the actual program. Let's assume it doesn't touch any files on disk (no opening sockets either). You would need to at least 1. Halt the process (SIGSTOP comes to mind) 2. Create a copy of the running program and /proc/$pid - which will also include memory and mmap details 3. Transfer everything to the other machine 4. Load memory, somehow spawn a spawn a new process with the info from /proc/$pid we saved, mmap the loaded memory into it 5. Continue the process on the new machine (SIGCONT) Let me admit that I do not have the slightest clue how to achieve step 4. I wonder if a systemd namespace could make things easier.
bravura 2y ago
This is an increasingly important area, with LLM generated code, and am curious about people's experiences with codesandbox vs e2b vs daytona
- nkko 2y ago
  Check this out: Spinning 100 Agents in Daytona https://youtu.be/OFFFyfgO2ik https://youtu.be/OFFFyfgO2ik we should also soon release open source speed benchmark
hobofan 2y ago
[2022]
walrus01 2y ago
> "Virtual machines are often seen as slow, expensive, bloated and outdated. " By who, exactly? Citations needed
- awestroke 2y ago
  It's not an uncommon viewpoint. Especially if you compare them to containers.
- sofixa 2y ago
  By almost everyone who has done a comparison. VMs have a full OS that needs to be maintained (patched, upgraded when EOL, etc.). Hypervisors traditionally cost a metric crapton of money per core. Yes, Proxmox is pretty good, but it's the exception, not the norm. They're also relatively slow in spinning up new VMs (kind of by definition, it takes a lot of time to emulate a full blown replica of hardware vs just starting a process in a cgroup/jail). And most of all, VMs are just solving the wrong problem. You don't care about emulating hardware, you care about running some workload. Maybe it needs specific hardware or a virtual version of it, but more likely than not, it's a regular batch processor or API that can happily run in a container with almost none of the overhead of a full VM.
  - tryauuum 2y ago
    while you are correct in calling the VM startup slow compared to the container startup, reading "emulating hardware" burns my eyes modern VMs don't emulate hardware. When a VM has a hard drive or a network device there's no sophisticated code to trick VM into believing that this is real hardware. Virtio drivers are about VM writing data in a memory area and assuming it's written to the disk / sent to the network (because in the background hypervisor reads the same memory area and does the job)
    - sofixa 2y ago
      > modern VMs don't emulate hardware They provide pretend hardware which isn't really necessary.
comprev 2y ago
Needs [2022] in the title
CompuIves 2y ago
Oh wow! Unexpected and cool to see this post on Hacker News! Since then we have evolved our VM infra a bit, and I've written two more posts about this. First, we started cloning VMs using userfaultfd, which allows us to bypass the disk and let children read memory directly from parent VMs [1]. And we also moved to saving memory snapshots compressed. To keep VM boots fast, we need to decompress on the fly as VMs read from the snapshot, so we chunk up snapshots in 4kb-8kb pieces that are zstd compressed [2]. Happy to answer any questions here! [1]: https://codesandbox.io/blog/cloning-microvms-using-userfaultfd https://codesandbox.io/blog/cloning-microvms-using-userfault... [2]: https://codesandbox.io/blog/how-we-scale-our-microvm-infrastructure-using-low-latency-memory-decompression https://codesandbox.io/blog/how-we-scale-our-microvm-infrast...
- pulkitsh1234 2y ago
  Enjoyed reading all these and learnt a lot! Thanks for taking the time out to write the blogs!
- zekrioca 2y ago
  The example code present in the link is not available. Would you know where it went? Thanks, great article!
nimbius 2y ago
>Virtual machines are often seen as slow, expensive, bloated and outdated. by whom? I tend to loathe firecracker posts because theyre all just thinly veiled ads for Amazon services. Firecracker is not included in the standard linux KVM/QEMU duo and has sparse documentation. you cannot deploy a firecracker image like a traditional VM. in fact there are no tools to assist in creating a firecracker VM, and the filesystem for the VM must be EXT4. TL;DR: this is all fun stuff if youre 200% cloud, but most companies run a ton of on-prem vms as well.
- hhh 2y ago
  I was using ignite for a while to create firecracker vms, i think it’s called flintlock now. Ignite worked great when I was using it.
nodesocket 2y ago
Has anybody tried running ollama and Open WebUI in firecracker instead of full VMs? I assume this should work, but not sure about GPU (single and multi) passthrough.
- nkko 2y ago
  As far as I understand Firecracker, you can't do a GPU passthrough
phgn 2y ago
(2022)
Imustaskforhelp 2y ago
In the minecraft example video We are shown a person who quit the server and then the server stops and restarts (that 2 second clone of vm) but what if I have a service like lets say normal minecraft servers like hypixel or others, they can't hope for a 2 second delay. Maybe we would have to use proxies in that case. I am genuinely interested by this tech. Currently, I am much in favour of tinykvm and its snapshotting because its even lighter than firecracker(I think). I really like the dev behind tinykvm as well.
manish_gill 2y ago
They tried running minecraft, but I wonder if a similar (or better) cloning is possible for a mission critical workload - like a database consuming a huge amount of memory. Neon uses QEMU to achieve this for example: https://neon.tech/docs/reference/glossary#live-migration https://neon.tech/docs/reference/glossary#live-migration but is that the only way?
dang 2y ago
Related: We clone a running VM in 2 seconds - https://news.ycombinator.com/item?id=38651805 https://news.ycombinator.com/item?id=38651805 - Dec 2023 (10 comments)
mschuster91 2y ago
> How to handle network and IP duplicates on cloned VMs That is indeed what I would love to read the most! Because no matter what you do, it gets complex - if you tear down the network stack of the "old" VM, applications (like Minecraft) might be heading down into unstable territory when the listener socket disappears and the "new" VM has to go through the entire DHCP flow that may easily take a second or more, and if you just do the equivalent of S3 sleep (suspend to RAM), the first "new" VM will have everything working as expected but any further VM being spawned from the template will run into duplicate IP/MAC address usage.
lofaszvanitt 2y ago
What problem is this supposed to solve?

169 points by hunvreus 2y ago

Cool article! The stack (and results) are impressive, but I also appreciate the article in itself, starting from basics and getting to the point in a clear and slowly expanding way. Easy to follow and appreciate. On a bit of a tangent rant, this kind of writing is slowly going away, taken over by LLM slop (and I'm a huge fan of LLMs, just not the people who write those kinds of articles). I was recently looking for real world benchmarks for vllm/sglang deployments of DeepSeek3 on a 8x 96GB pod, to see if the model fits into the amount of RAM, with kv cache and context length, what numbers to people get, etc. Of the ~20 articles that google surfaced on various attempts of keywords, none were what I was looking for. The excerpts seemed promising, some even offered tables & stuff related to ds3 and RAM usage, but all were LLM crap. All were written in that simple style - intro - bla bla - conclusion, some even had RAM requirements that made no sense (running a model trained in FP8 in 16bit, something noone would do, etc.)

fxtentacle 2y ago
While I fully agree with you on the absence of good benchmarks and the growing LLM slop ... "running a model trained in FP8 in 16bit, something noone would do, etc" I did that because on the RTX 3090 - which can be a good bang per buck for inference - the FP8 support is nerfed at the driver level. So a kernel that upscales FP8 to FP16 inside SRAM, then does the matmul, then downscales to FP8 again can bring massive performance benefits on those consumer cards. BTW, you can run a good DeepSeek3 quant on a single H200.
- NitpickLawyer 2y ago
  Thanks! I was looking at blackwell 6000PROs, 8x 96GB for running full fp8 (as it's supported and presumably fast). I know AWQ should run, and be pretty snappy & efficient w/ the new MLA added, but wanted to check if fp8 fits as well, because from a simple napkin math it seems pretty tight (might only work for bs1, ctx_len <8k which would probably not be suited for coding tasks).
thomasjudge 2y ago
You are describing this as a writing problem, but it sounds more like a search results/search engine problem

1970-01-01 2y ago

The VM vs container debate is fascinating. They are separate yet slowly merging concepts that are becoming more blurred as technology becomes cheaper and faster. If the real bottleneck to scale is adaptable code, then it is foolish to dismiss the VM as outdated tech when it can be completely rehomed in 2 seconds. That megabyte of python code managing your containers would still be busy checking it's dependencies in that same timeframe.

znpy 2y ago
In a way it's nothing new. MOSIX/openMosix (https://en.wikipedia.org/wiki/MOSIX https://en.wikipedia.org/wiki/MOSIX, https://en.wikipedia.org/wiki/OpenMosix https://en.wikipedia.org/wiki/OpenMosix) did similar stuff with individual processes. It would probably be even faster as then you would have to only move process memory and its state rather than the whole VM memory (and it state). I guess it would/could be nice to have something that moves kubernetes pods around rather killing them and starting new ones.
jiggawatts 2y ago
IMHO the real advancement was the implicit snapshot Docker takes after every line in a Dockerfile. Virtual Machine builder scripts like Packer could do this… but don’t. It’s a choice, not an inherent technology limitation.
- dontlaugh 2y ago
  I have the opposite opinion, the implicit overlay filesystem in Docker is an unnecessary and frustrating complication.

simonklitj 2y ago

Interesting read—thanks! One question: in the CoW example, if VM A modifies the data post-fork, what does VM B see when it later copies that data? Does it get the original data from the time of the fork, or VM A’s modified version?

CompuIves 2y ago
I talk a bit about this here: https://codesandbox.io/blog/cloning-microvms-using-userfaultfd https://codesandbox.io/blog/cloning-microvms-using-userfault.... Before VM A updates its data, the data is copied over to VM B if VM B hadn't written/read that data yet.
- simonklitj 2y ago
  clever! Thank you.

londons_explore 2y ago

Unmentioned: there are serious security issues with memory cloning code not designed for it. For example, an SSL library might have pre-calculated the random nonce for the next incoming SSL connection. If you clone the VM containing a process using that library, now both child VM's will use the same nonce. Some crypto is 100% broken open if a nonce is reused.

generalizations 2y ago
Sounds like it would simply be inappropriate to clone & use a VM that's assuming it's data is unique. This would also be true of other conditions, e.g. if you needed to spoof a MAC or IPv6 address & picked one randomly.
- londons_explore 2y ago
  The problem is modern software is so fiendishly complicated there almost certainly is stuff like that in the code. The question is where, and does it matter?
  - generalizations 2y ago
    And the last question is, can the parts with stuff like that be extracted from the rest and run separately?
hypeatei 2y ago
> might have pre-calculated the random nonce Isn't this still a concern even if you're not pre-calculating way ahead of time? If you generate it when needed, it could still catch you at the wrong time (e.g. right before encryption, but right after nonce generation)
- zamadatix 2y ago
  Unless your encryption and transport protocols are 100% stateless only 1 connection will actually be able to form, even if you duplicate the machine during connection creation. The problem with pre-computing a bunch and keeping them in memory is brand new connections made post cloning would use the same list of nonces.

pragma_x 2y ago

I'm starting to see a pattern here. This describes a technology that rapidly deploys "VM" instances in the cloud which support things like Lambda and single-process containers. At what point do we scale this all back to a more rudimentary OS that provides security and process management across multiple physical machines? Or is there already a Linux distro that does this? I ask because watching cloud providers like AWS slowly reinvent mainframes just seems like the painful way around.

no_wizard 2y ago
EDIT: leaving the answer, but I simply misinterpreted what they meant. This isn't the same thing BSD has had jails for a long time, which let you achieve isolation on a system in this manner, or at least close to it.
- hedora 2y ago
  They’re missing multi-machine orchestration: Run thousands of jails on these dozen machines. Don’t bother me with the details at runtime. They are also missing an ergonomic tool like dockerfiles. The following file, plus a cli tool for “run N copies on my M machines” should be enough to run bsd in prod, and it is not: “FROM openbsd:latest ; CMD pkg -i apache ; echo “apache=enabled >> /etc/rc.defaults ; COPY public_html /var/www/ ; CMD init” I don’t think writing the tooling would be that difficult, but it was missing the last time I looked.
  - no_wizard 2y ago
    I think I may have simply misinterpreted what you meant. You're right, its not Dockerfile-esque easy
jerf 2y ago
We've been cycling around that wheel for a while. If there's any difference now versus the past, it is that I think right now pretty much every point on the wheel is available quite readily now. If you want a more "rudimentary OS" you don't need to wait for the next turning of the wheel, it's here now. Need full VMs? Still a practical technology. Containers enough? Actively in development and use. Mix & match? Around any sensible combination you can do it now. And so on.

mystraline 2y ago

Different proposal: Let's say we have 2 Linux machines. Identical hardware, identical libs. I'd like to run a simple program on one machine, and then during mid-calculation, would like to transfer the running program to the other machine. Is this doable?

new_user_final 2y ago
Unrelated, but somewhat similar in higher level, you can transfer state with durable execution, e.g temporal.io. Instead of RAM, program's state saved in DB and execution environment resume in the previous state when restarted
- WJW 2y ago
  How does such a method retain things like open network connections that have significant kernel state involved as well?
  - dilyevsky 2y ago
    it does not. all the state that you need to make "durable" needs to be explicitly committed in temporal via their sdk
dilyevsky 2y ago
Yes - using Criu[0] or docker checkpoint/restore api (which uses criu) [0] -https://criu.org/Main_Page https://criu.org/Main_Page
panki27 2y ago
Interesting thought, but highly dependant on the actual program. Let's assume it doesn't touch any files on disk (no opening sockets either). You would need to at least 1. Halt the process (SIGSTOP comes to mind) 2. Create a copy of the running program and /proc/$pid - which will also include memory and mmap details 3. Transfer everything to the other machine 4. Load memory, somehow spawn a spawn a new process with the info from /proc/$pid we saved, mmap the loaded memory into it 5. Continue the process on the new machine (SIGCONT) Let me admit that I do not have the slightest clue how to achieve step 4. I wonder if a systemd namespace could make things easier.

bravura 2y ago

This is an increasingly important area, with LLM generated code, and am curious about people's experiences with codesandbox vs e2b vs daytona

nkko 2y ago
Check this out: Spinning 100 Agents in Daytona https://youtu.be/OFFFyfgO2ik https://youtu.be/OFFFyfgO2ik we should also soon release open source speed benchmark

hobofan 2y ago

[2022]

walrus01 2y ago

> "Virtual machines are often seen as slow, expensive, bloated and outdated. " By who, exactly? Citations needed

awestroke 2y ago
It's not an uncommon viewpoint. Especially if you compare them to containers.
sofixa 2y ago
By almost everyone who has done a comparison. VMs have a full OS that needs to be maintained (patched, upgraded when EOL, etc.). Hypervisors traditionally cost a metric crapton of money per core. Yes, Proxmox is pretty good, but it's the exception, not the norm. They're also relatively slow in spinning up new VMs (kind of by definition, it takes a lot of time to emulate a full blown replica of hardware vs just starting a process in a cgroup/jail). And most of all, VMs are just solving the wrong problem. You don't care about emulating hardware, you care about running some workload. Maybe it needs specific hardware or a virtual version of it, but more likely than not, it's a regular batch processor or API that can happily run in a container with almost none of the overhead of a full VM.
- tryauuum 2y ago
  while you are correct in calling the VM startup slow compared to the container startup, reading "emulating hardware" burns my eyes modern VMs don't emulate hardware. When a VM has a hard drive or a network device there's no sophisticated code to trick VM into believing that this is real hardware. Virtio drivers are about VM writing data in a memory area and assuming it's written to the disk / sent to the network (because in the background hypervisor reads the same memory area and does the job)
  - sofixa 2y ago
    > modern VMs don't emulate hardware They provide pretend hardware which isn't really necessary.

comprev 2y ago

Needs [2022] in the title

CompuIves 2y ago

Oh wow! Unexpected and cool to see this post on Hacker News! Since then we have evolved our VM infra a bit, and I've written two more posts about this. First, we started cloning VMs using userfaultfd, which allows us to bypass the disk and let children read memory directly from parent VMs [1]. And we also moved to saving memory snapshots compressed. To keep VM boots fast, we need to decompress on the fly as VMs read from the snapshot, so we chunk up snapshots in 4kb-8kb pieces that are zstd compressed [2]. Happy to answer any questions here! [1]: https://codesandbox.io/blog/cloning-microvms-using-userfaultfd https://codesandbox.io/blog/cloning-microvms-using-userfault... [2]: https://codesandbox.io/blog/how-we-scale-our-microvm-infrastructure-using-low-latency-memory-decompression https://codesandbox.io/blog/how-we-scale-our-microvm-infrast...

pulkitsh1234 2y ago
Enjoyed reading all these and learnt a lot! Thanks for taking the time out to write the blogs!
zekrioca 2y ago
The example code present in the link is not available. Would you know where it went? Thanks, great article!

nimbius 2y ago

>Virtual machines are often seen as slow, expensive, bloated and outdated. by whom? I tend to loathe firecracker posts because theyre all just thinly veiled ads for Amazon services. Firecracker is not included in the standard linux KVM/QEMU duo and has sparse documentation. you cannot deploy a firecracker image like a traditional VM. in fact there are no tools to assist in creating a firecracker VM, and the filesystem for the VM must be EXT4. TL;DR: this is all fun stuff if youre 200% cloud, but most companies run a ton of on-prem vms as well.

hhh 2y ago
I was using ignite for a while to create firecracker vms, i think it’s called flintlock now. Ignite worked great when I was using it.

nodesocket 2y ago

Has anybody tried running ollama and Open WebUI in firecracker instead of full VMs? I assume this should work, but not sure about GPU (single and multi) passthrough.

nkko 2y ago
As far as I understand Firecracker, you can't do a GPU passthrough

phgn 2y ago

(2022)

Imustaskforhelp 2y ago

In the minecraft example video We are shown a person who quit the server and then the server stops and restarts (that 2 second clone of vm) but what if I have a service like lets say normal minecraft servers like hypixel or others, they can't hope for a 2 second delay. Maybe we would have to use proxies in that case. I am genuinely interested by this tech. Currently, I am much in favour of tinykvm and its snapshotting because its even lighter than firecracker(I think). I really like the dev behind tinykvm as well.

manish_gill 2y ago

They tried running minecraft, but I wonder if a similar (or better) cloning is possible for a mission critical workload - like a database consuming a huge amount of memory. Neon uses QEMU to achieve this for example: https://neon.tech/docs/reference/glossary#live-migration https://neon.tech/docs/reference/glossary#live-migration but is that the only way?

dang 2y ago

Related: We clone a running VM in 2 seconds - https://news.ycombinator.com/item?id=38651805 https://news.ycombinator.com/item?id=38651805 - Dec 2023 (10 comments)

mschuster91 2y ago

> How to handle network and IP duplicates on cloned VMs That is indeed what I would love to read the most! Because no matter what you do, it gets complex - if you tear down the network stack of the "old" VM, applications (like Minecraft) might be heading down into unstable territory when the listener socket disappears and the "new" VM has to go through the entire DHCP flow that may easily take a second or more, and if you just do the equivalent of S3 sleep (suspend to RAM), the first "new" VM will have everything working as expected but any further VM being spawned from the template will run into duplicate IP/MAC address usage.

lofaszvanitt 2y ago

What problem is this supposed to solve?