4 ms·

Azure, AWS and GCP all have live migration. VMWare has it too.

free652 4y ago
Are you sure, because AWS consistently requires me to migrate to a different host. They go as far as shutting down instances, but don't do any kind of live migrations.
dilyevsky 4y ago
Ec2 does not have live migration. On azure it’s spotty so not every maintenance can offer it.
- deleted 4y ago
  [deleted]
- ta20200710 4y ago
  EC2 does support live migration, but it's not public and only for certain instance types/hypervisors. See: https://news.ycombinator.com/item?id=17815806 https://news.ycombinator.com/item?id=17815806
  - _msw_ 4y ago
    Here's a comment that I made in a past thread. https://news.ycombinator.com/item?id=26650082 https://news.ycombinator.com/item?id=26650082
    - dilyevsky 4y ago
      My experience running c5/6 instances makes me very confident ec2 doesn’t do live migration for these. Fwiw gcp live migration on latency sensitive workloads is very noticeable and often time straight up causes instance crash
      - dwmw2 4y ago
        Intrigued by this observation. What is it about your experience that leads you to conclude that EC2 doesn't do live migration? And could it be phrased differently as "EC2 doesn't do live migration badly"?
        dilyevsky 4y ago
        Mainly the barrage of "instance hardware degradation" emails that i get whereas on gcp those are just migrated (sometimes with a reboot/crash). Also there is no brownout. I've never used t2/3s which apparently do support migration which would make sense.
        my123 4y ago
        After some kinds of hardware failure, it can become impossible to do live migration safely. When a crash can ensure due to a live migration from faulty HW, I'd argue that it's much better to not attempt it.
outworlder 4y ago
Not really. Or at least not in the same league. AWS doesn't have live migration at all. You have to stop/start. Azure technically does, but it doesn't always work(they say 90%). 30 seconds is a long time. VMWare has live migration (and seems to be the closest to what GCP does) but it is still an inferior user experience. This is the key thing you are missing – GCP not only has live migration, but it is completely transparent. We do not have to initiate migration. GCP does, transparently, 100% of the time. We have never even notice migrations even when we were actively watching those instances. We don't know or care what hypervisors are involved. They even preserve the network connections. https://cloudplatform.googleblog.com/2015/03/Google-Compute-Engine-uses-Live-Migration-technology-to-service-infrastructure-without-application-downtime.html https://cloudplatform.googleblog.com/2015/03/Google-Compute-...
- jiggawatts 4y ago
  VMware's live migration is totally seamless, so I don't know what you mean by "inferior user experience". You typically see less than a second of packet loss, and a small performance hit for about a minute while the memory is "swapped" across to the new machine. Similarly, VMware has had live storage migration for years. VMware is lightyears ahead of the big clouds, but unfortunately they "missed the boat" on the public cloud, despite having superior foundational technology. For example: - A typical vSphere cluster would use live migration to balance workloads dynamically. You don't notice this as an end user, but it allows them to bin-pack workloads up above 80% CPU utilisation in my experience with good results. (Especially if you allocate priorities, min/max limits, etc...) - You can version-upgrade a vSphere cluster live. This includes rolling hypervisor kernel upgrades and live disk format changes. The upgrade wizard is a fantastic thing that asks only for the cluster controller name and login details! Click "OK" and watch the progress bar. - Flexible keep-apart and keep-together rules that can updated at any time, and will take effect via live migration. This is sort-of like the Kubernetes "control loops", but the migrations are live and memory-preserving instead of stop-start like with containers. - Online changes to virtual hardware, including adding not just NICs and disks, but also CPU and memory! - Thin-provisioned disks, and memory deduplication for efficiencies approaching that of containerisation. - Flexible snapshots, including the ability for "thin provisioned" virtual machines to share a base snapshot. This is often used for virtual desktops or terminal services, and again this approaches containerisation in terms of cloning speed and storage efficiency. In other words, VMware had all of the pieces, and just... didn't... use it to make a public cloud. We could have had "cloud.vmware.com" or whatever 15 years ago, but they decided to slowly jack up the price on their enterprise customers instead. For comparison, in Azure: You can't add a VM to an availability set (keep apart rule) or remove the VM from it without a stop-start cycle. You can't make most changes (SKU, etc...) to a VM in an availability set without turning off every machine in the same AS! This is just one example of many where the public cloud has a "checkbox" availability feature that actually decreases availability. For a long time, changing an IP address in AWS required the VM to be basically blown away and recreated. That brought back memories of the Windows NT 4 days in 1990s when an IP change required a reboot cycle.

Are you sure, because AWS consistently requires me to migrate to a different host. They go as far as shutting down instances, but don't do any kind of live migrations.

dilyevsky 4y ago

Ec2 does not have live migration. On azure it’s spotty so not every maintenance can offer it.

deleted 4y ago
[deleted]
ta20200710 4y ago
EC2 does support live migration, but it's not public and only for certain instance types/hypervisors. See: https://news.ycombinator.com/item?id=17815806 https://news.ycombinator.com/item?id=17815806
- _msw_ 4y ago
  Here's a comment that I made in a past thread. https://news.ycombinator.com/item?id=26650082 https://news.ycombinator.com/item?id=26650082
  - dilyevsky 4y ago
    My experience running c5/6 instances makes me very confident ec2 doesn’t do live migration for these. Fwiw gcp live migration on latency sensitive workloads is very noticeable and often time straight up causes instance crash
    - dwmw2 4y ago
      Intrigued by this observation. What is it about your experience that leads you to conclude that EC2 doesn't do live migration? And could it be phrased differently as "EC2 doesn't do live migration badly"?
      - dilyevsky 4y ago
        Mainly the barrage of "instance hardware degradation" emails that i get whereas on gcp those are just migrated (sometimes with a reboot/crash). Also there is no brownout. I've never used t2/3s which apparently do support migration which would make sense.
        my123 4y ago
        After some kinds of hardware failure, it can become impossible to do live migration safely. When a crash can ensure due to a live migration from faulty HW, I'd argue that it's much better to not attempt it.

outworlder 4y ago

Not really. Or at least not in the same league. AWS doesn't have live migration at all. You have to stop/start. Azure technically does, but it doesn't always work(they say 90%). 30 seconds is a long time. VMWare has live migration (and seems to be the closest to what GCP does) but it is still an inferior user experience. This is the key thing you are missing – GCP not only has live migration, but it is completely transparent. We do not have to initiate migration. GCP does, transparently, 100% of the time. We have never even notice migrations even when we were actively watching those instances. We don't know or care what hypervisors are involved. They even preserve the network connections. https://cloudplatform.googleblog.com/2015/03/Google-Compute-Engine-uses-Live-Migration-technology-to-service-infrastructure-without-application-downtime.html https://cloudplatform.googleblog.com/2015/03/Google-Compute-...

jiggawatts 4y ago
VMware's live migration is totally seamless, so I don't know what you mean by "inferior user experience". You typically see less than a second of packet loss, and a small performance hit for about a minute while the memory is "swapped" across to the new machine. Similarly, VMware has had live storage migration for years. VMware is lightyears ahead of the big clouds, but unfortunately they "missed the boat" on the public cloud, despite having superior foundational technology. For example: - A typical vSphere cluster would use live migration to balance workloads dynamically. You don't notice this as an end user, but it allows them to bin-pack workloads up above 80% CPU utilisation in my experience with good results. (Especially if you allocate priorities, min/max limits, etc...) - You can version-upgrade a vSphere cluster live. This includes rolling hypervisor kernel upgrades and live disk format changes. The upgrade wizard is a fantastic thing that asks only for the cluster controller name and login details! Click "OK" and watch the progress bar. - Flexible keep-apart and keep-together rules that can updated at any time, and will take effect via live migration. This is sort-of like the Kubernetes "control loops", but the migrations are live and memory-preserving instead of stop-start like with containers. - Online changes to virtual hardware, including adding not just NICs and disks, but also CPU and memory! - Thin-provisioned disks, and memory deduplication for efficiencies approaching that of containerisation. - Flexible snapshots, including the ability for "thin provisioned" virtual machines to share a base snapshot. This is often used for virtual desktops or terminal services, and again this approaches containerisation in terms of cloning speed and storage efficiency. In other words, VMware had all of the pieces, and just... didn't... use it to make a public cloud. We could have had "cloud.vmware.com" or whatever 15 years ago, but they decided to slowly jack up the price on their enterprise customers instead. For comparison, in Azure: You can't add a VM to an availability set (keep apart rule) or remove the VM from it without a stop-start cycle. You can't make most changes (SKU, etc...) to a VM in an availability set without turning off every machine in the same AS! This is just one example of many where the public cloud has a "checkbox" availability feature that actually decreases availability. For a long time, changing an IP address in AWS required the VM to be basically blown away and recreated. That brought back memories of the Windows NT 4 days in 1990s when an IP change required a reboot cycle.