4 ms·

Unclear what the article has to do with reliability. Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. Azure feels like I'

Unclear what the article has to do with reliability. Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. Azure feels like I'm starting a Boeing 747 instead of a VM. However, there's one aspect where GCP is a clear winner on the reliability front. They auto-migrate instances transparently and with close to zero impact to workloads – I want to say zero impact but it's not technically zero. In comparison, in AWS you need to stop/start your instance yourself so that it will move to another hypervisor(depending on the actual issue AWS may do it for you). That definitely has impact on your workloads. We can sometimes architect around it but there's still something to worry about. Given the number of instances we run, we have multiple machines to deal with weekly. We get all these 'scheduled maintenance' events (which sometimes aren't really all that scheduled), with some instance IDs(they don't even bother sending the name tag), and we have to deal with that. I already thought stop/start was an improvement on tech at the time (Openstack, for example, or even VMWare) just because we don't have to think about hypervisors, we don't have to know, we don't care. We don't have to ask for migrations to be performed, hypervisors are pretty much stateless. However, on GCP? We had to stop/start instances exactly zero times, out of the thousands we run and have been running for years. We can see auto-migration events when we bother checking the logs. Otherwise, we don't even notice the migration happened. It's pretty old tech too: https://cloudplatform.googleblog.com/2015/03/Google-Compute-Engine-uses-Live-Migration-technology-to-service-infrastructure-without-application-downtime.html https://cloudplatform.googleblog.com/2015/03/Google-Compute-...

willcipriano 4y ago
I always wondered why you couldn't do that on AWS, mainly because I could do it at home with Hyper-V a decade ago. https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/microsoft-hyper-v-server-2008-R2/ee815293(v=ws.10) https://learn.microsoft.com/en-us/previous-versions/windows/...
voidfunc 4y ago
> Azure feels like I'm starting a Boeing 747 instead of a VM. Huh... interesting, this has not been my experience with Azure VM launch times. I'm usually surprised how quickly they pop up.
- jiggawatts 4y ago
  Depends on your disks. Premium SSD allows 30 minutes of "burst" IOPS, which can bring down boot times to about 2-5 seconds for a typical Windows VM. The provisioning time is a further 60-180 seconds on top. (The fastest I could get it is about 40 seconds using a "smalldisk" image to ephemeral storage, but then it took a further 30 seconds or so for the VM to become available.) Standard HDD was slow enough that the boot phase alone would take minutes, and then the VM provisioning time is almost irrelevant in comparison.
jcheng 4y ago
> Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. FWIW this article is saying the opposite--it's AWS that beats GCP in startup speed.
- valleyjo 4y ago
  This article states that GPU instances are slower on GCP - it doesn’t make any claims about non-GPU instances.
yolovoe 4y ago
EC2 live migrates instances too. Not sure where we are with rollout across the fleet. The reason, from what I understand, why GCP does live migration more is because ec2 focused on live updates instead of live migration. Whereas GCP migrates instances to update servers, ec2 live updates everything down to firmware while instances are running. Curious, what instance types are you using on EC2 that you see so much maintenance?
- outworlder 4y ago
  > Curious, what instance types are you using on EC2 that you see so much maintenance We use a bunch of different types. M5 and R5 (different sizes) are the most commonly used types but we use many different families. I haven't done an analysis to figure out which types are hotspots. This is across thousands of instances over many regions worldwide. The percentage is low, but that still translates to daily maintenance alerts.

willcipriano 4y ago

I always wondered why you couldn't do that on AWS, mainly because I could do it at home with Hyper-V a decade ago. https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/microsoft-hyper-v-server-2008-R2/ee815293(v=ws.10) https://learn.microsoft.com/en-us/previous-versions/windows/...

voidfunc 4y ago

> Azure feels like I'm starting a Boeing 747 instead of a VM. Huh... interesting, this has not been my experience with Azure VM launch times. I'm usually surprised how quickly they pop up.

jiggawatts 4y ago
Depends on your disks. Premium SSD allows 30 minutes of "burst" IOPS, which can bring down boot times to about 2-5 seconds for a typical Windows VM. The provisioning time is a further 60-180 seconds on top. (The fastest I could get it is about 40 seconds using a "smalldisk" image to ephemeral storage, but then it took a further 30 seconds or so for the VM to become available.) Standard HDD was slow enough that the boot phase alone would take minutes, and then the VM provisioning time is almost irrelevant in comparison.

jcheng 4y ago

> Yes, spinning up machines on GCP is incredibly fast and has always been. AWS is decent. FWIW this article is saying the opposite--it's AWS that beats GCP in startup speed.

valleyjo 4y ago
This article states that GPU instances are slower on GCP - it doesn’t make any claims about non-GPU instances.

yolovoe 4y ago

EC2 live migrates instances too. Not sure where we are with rollout across the fleet. The reason, from what I understand, why GCP does live migration more is because ec2 focused on live updates instead of live migration. Whereas GCP migrates instances to update servers, ec2 live updates everything down to firmware while instances are running. Curious, what instance types are you using on EC2 that you see so much maintenance?

outworlder 4y ago
> Curious, what instance types are you using on EC2 that you see so much maintenance We use a bunch of different types. M5 and R5 (different sizes) are the most commonly used types but we use many different families. I haven't done an analysis to figure out which types are hotspots. This is across thousands of instances over many regions worldwide. The percentage is low, but that still translates to daily maintenance alerts.