4 ms·

How does migrating a vm to another physical machine work?

• rejectfinite 4y ago
  vsphere vmotion has been a thing for years lmao
• roomey 4y ago
  VMware has been doing this for years, it's called vmotion and there is a lot of documentation about it if you are interested (eg https://www.thegeekpub.com/8407/how-vmotion-works/ https://www.thegeekpub.com/8407/how-vmotion-works/ ) Essential, memory state is copied to the new host, the VM is stunned for a millisecond and the cpu states is copied and resumed on the new host (you may see a dropped ping). All the networking and storage is virtual anyway so that is "moved" (it's not really moved) in the background.
  • mh- 4y ago
    Up to 500ms per your source, depending on how much churn there is in the memory from the source system. Very cool.
  • davidro80 4y ago
  • lbhdc 4y ago
    That is really interesting I didn't realize it was so fast. Thanks for the post I will give it a read!
  • jiggawatts 4y ago
    The clever trick here is that they'll pre-copy most of the memory without bothering to do it consistently, but mark pages that the source had written to as "dirty". The network cutover is stop-the-world, but VMware doesn't copy the dirty pages during the stop. Instead, it simply treats them as "swapped to pagefile", where the pagefile is actually the source machine memory. When computation resumes at the target, the source is used to page memory back in on-demand. This allows very fast cutovers.
  • politelemon 4y ago
    > VM is stunned for a millisecond This conjures up hilarious mental imagery, thanks
    • kevincox 4y ago
      You just bop it on the head, and move it to the new machine quickly. By the time the VM comes to it won't even realize that it is in a new home.
• valleyjo 4y ago
  Stream the contents of ram from source to dest, pause the source, reprogram the network and copy and memory that changed since the initial stream, resume the dest, destroy the source, profit.
• pclmulqdq 4y ago
  They pause your VM, copy everything about its state over to the new machine, and quickly start the other instance. It's pretty clever. I think there are tricks you can play with machines that have large memory footprints to copy most of it before the pause, and only copy what has changed since then during the pause. The disks are all on the network, so no need to move anything there.
  • prmoustache 4y ago
    In reality it sync the memory first to the other host and only pause the vm when the last state sync is small enough to be so quick the pause is barely measurable.
    • water-your-self 4y ago
      Indian jones and the register states
    • lbhdc 4y ago
      When its transferring the state to the target, how does it handle memory updates that are happening at that time? Is the programs execution paused at that point?
      • outworlder 4y ago
        No, they keep track of dirty pages.
      • GauntletWizard 4y ago
        No, but the memory accesses have hooks that say "This memory was written". Then, program execution is paused, and just the sections of memory that were written are copied again. This has memory performance characteristics - I ran a benchmark of memory read/write speed while this was happening once. It more than halved memory speed for the 30s or so it took from migration started to migration complete. The pause, too, was much longer.
        • 4y ago
• the_duke 4y ago
  This blog post is pretty old (2015) but gives a good introduction. https://cloudplatform.googleblog.com/2015/03/Google-Compute-Engine-uses-Live-Migration-technology-to-service-infrastructure-without-application-downtime.html?m=1 https://cloudplatform.googleblog.com/2015/03/Google-Compute-...
  • lbhdc 4y ago
    Thanks for sharing, I will give it a read!