4 ms·

Google Cloud now lets you suspend and resume VMs

78 points by akrymski 4y ago

akrymski 4y ago
It’s worth noting that Suspend/Resume doesn’t work for GPU instances. Surprising given that NVidia has vGPUs capable of hibernation: https://docs.nvidia.com/grid/15.0/grid-vgpu-release-notes-microsoft-azure-stack-hci/ https://docs.nvidia.com/grid/15.0/grid-vgpu-release-notes-mi... "vGPU hibernation is supported on all supported GPUs, Microsoft Azure Stack HCI hypervisor software releases, and guest operating systems."
- strstr 4y ago
  GCE passes through GPU hardware instead of using vGPU. (I believe, but not 100% certain this is universal for all GPU types.)
_8j50 4y ago
If only you could take and restore snapshots easily in this platforms. I use used nested virtualization with qemu or vbox to avoid having service accounts, automation scripts, functions,etc... But this is a great step forward, I have seen (including with myself) heavy usage of cloud VMs as workstations.
cwilkes 4y ago
[ 2022 ]
ericpauley 4y ago
Note: AWS has supported this for many years: https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-instances/ https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-inst... I do wonder how big the use case for suspend/hibernate is in practice. Are there that many services that (a) see infrequent use that justifies this cost reduction and (b) can't just be booted fresh when needed. I could see if there's a lot of compute-intensive data cached, but this could also just be stored in a disk-persistent cache.
- bradhe 4y ago
  It’s a useful feature in some cases for certain stateful workloads (e.g. cloud-based workstations) but it’s almost always 100% better architecturally to develop systems that can just be thrown away and rebuilt vs needing to preserve state.
  - ericpauley 4y ago
    True. We do a fair bit of cloud-based dev environments and I’d be worried about risks from tasks getting paused/resumed that aren’t well-designed for it. In theory this could cause undesirable outcomes but it’s probably not an issue in practice.
- OkayPhysicist 4y ago
  I've used it extensively for hosting the heavily-modded Minecraft servers we play with. Those things can take easily 5-10 minutes to boot up fresh, as opposed to the very quick resume.
- outworlder 4y ago
  > Note: AWS has supported this for many years: https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-inst https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-inst... We have been using GCP hibernation since 2018 too, but it was alpha back then. The AWS version also had some limitations (it only supposed Amazon Linux at the time) - current prerequisites here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/hibernating-prerequisites.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/hibernat... We have hundreds of developer K8s clusters that get used during a normal workday and get shutdown later. Because there are many services (each with their own readiness criteria), a cold boot takes a long time until all services stabilize (worst case I've seen was over 30 minutes). With hibernation they can resume where they started off within a couple of minutes. During development it's often the case that one may be doing experiments and changing the current state, so preserving everything is a plus. If, as it was often the case when it was alpha, hibernation failed, then the normal startup procedure would take place. We don't use it in production but I don't think it's inconceivable that there are other use-cases where preserving state could shave time in getting resources ready.
justsomeuser 4y ago
How long does it take to resume a VM?
Temporary_31337 4y ago
Checking which year is this as VMWare, Virtual Box etc have supported this for more than a decade…
- CobrastanJorji 4y ago
  Checking the year is a good idea, as this is an article from exactly one year ago today.
- oceanplexian 4y ago
  It's been a decade since I used VMWare and I'm still waiting for Google, Amazon, etc. to catch up (Live Migration, Fault Tolerance, Instant Clone). Unfortunately all we've gotten is 15 different ways to run a docker container.
nerpderp82 4y ago
This is old. Suspension isn't transparent to the VM which reduces its usefulness. > When you make a suspend request, you send an ACPI suspend signal to the VM instance. If the VM does not respond to the ACPI S3 suspend signal within a couple of minutes, Compute Engine cancels the suspend attempt and returns the VM to a RUNNING state.
- outworlder 4y ago
  Same as AWS. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-hibernate-overview.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance... > When you initiate hibernation, the instance moves to the stopping state. Amazon EC2 signals the operating system to perform hibernation (suspend-to-disk). The hibernation freezes all of the processes, saves the contents of the RAM to the EBS root volume, and then performs a regular shutdown.
  - nerpderp82 4y ago
    I believe under VMWare a suspension is transparent to the OS. The hypervisor just stops running the guest.
  - tecleandor 4y ago
    I guess that time keeping could go crazy if you don't signal the OS to suspend or something like that. I remember some instances (years ago) where a VMWare live migration would make a VM clock jump or do something weird, and the guest OS would... "get scared".
outworlder 4y ago
We have been using this feature as 'beta' for a few years now with no issues. I've reported some bugs to GCP folks back when it was in an alpha state.
7373737373 4y ago
I wish more programming languages had this - would make it much easier to program simulations because agents could be written in the same language, instead of some other preemptible DSL, it'd also enable reproducible debugging and "system-jumping" processes if the suspended state could be serialized

78 points by akrymski 4y ago

It’s worth noting that Suspend/Resume doesn’t work for GPU instances. Surprising given that NVidia has vGPUs capable of hibernation: https://docs.nvidia.com/grid/15.0/grid-vgpu-release-notes-microsoft-azure-stack-hci/ https://docs.nvidia.com/grid/15.0/grid-vgpu-release-notes-mi... "vGPU hibernation is supported on all supported GPUs, Microsoft Azure Stack HCI hypervisor software releases, and guest operating systems."

strstr 4y ago
GCE passes through GPU hardware instead of using vGPU. (I believe, but not 100% certain this is universal for all GPU types.)

_8j50 4y ago

If only you could take and restore snapshots easily in this platforms. I use used nested virtualization with qemu or vbox to avoid having service accounts, automation scripts, functions,etc... But this is a great step forward, I have seen (including with myself) heavy usage of cloud VMs as workstations.

cwilkes 4y ago

[ 2022 ]

ericpauley 4y ago

Note: AWS has supported this for many years: https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-instances/ https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-inst... I do wonder how big the use case for suspend/hibernate is in practice. Are there that many services that (a) see infrequent use that justifies this cost reduction and (b) can't just be booted fresh when needed. I could see if there's a lot of compute-intensive data cached, but this could also just be stored in a disk-persistent cache.

bradhe 4y ago
It’s a useful feature in some cases for certain stateful workloads (e.g. cloud-based workstations) but it’s almost always 100% better architecturally to develop systems that can just be thrown away and rebuilt vs needing to preserve state.
- ericpauley 4y ago
  True. We do a fair bit of cloud-based dev environments and I’d be worried about risks from tasks getting paused/resumed that aren’t well-designed for it. In theory this could cause undesirable outcomes but it’s probably not an issue in practice.
OkayPhysicist 4y ago
I've used it extensively for hosting the heavily-modded Minecraft servers we play with. Those things can take easily 5-10 minutes to boot up fresh, as opposed to the very quick resume.
outworlder 4y ago
> Note: AWS has supported this for many years: https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-inst https://aws.amazon.com/blogs/aws/new-hibernate-your-ec2-inst... We have been using GCP hibernation since 2018 too, but it was alpha back then. The AWS version also had some limitations (it only supposed Amazon Linux at the time) - current prerequisites here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/hibernating-prerequisites.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/hibernat... We have hundreds of developer K8s clusters that get used during a normal workday and get shutdown later. Because there are many services (each with their own readiness criteria), a cold boot takes a long time until all services stabilize (worst case I've seen was over 30 minutes). With hibernation they can resume where they started off within a couple of minutes. During development it's often the case that one may be doing experiments and changing the current state, so preserving everything is a plus. If, as it was often the case when it was alpha, hibernation failed, then the normal startup procedure would take place. We don't use it in production but I don't think it's inconceivable that there are other use-cases where preserving state could shave time in getting resources ready.

justsomeuser 4y ago

How long does it take to resume a VM?

Temporary_31337 4y ago

Checking which year is this as VMWare, Virtual Box etc have supported this for more than a decade…

CobrastanJorji 4y ago
Checking the year is a good idea, as this is an article from exactly one year ago today.
oceanplexian 4y ago
It's been a decade since I used VMWare and I'm still waiting for Google, Amazon, etc. to catch up (Live Migration, Fault Tolerance, Instant Clone). Unfortunately all we've gotten is 15 different ways to run a docker container.

nerpderp82 4y ago

This is old. Suspension isn't transparent to the VM which reduces its usefulness. > When you make a suspend request, you send an ACPI suspend signal to the VM instance. If the VM does not respond to the ACPI S3 suspend signal within a couple of minutes, Compute Engine cancels the suspend attempt and returns the VM to a RUNNING state.

outworlder 4y ago
Same as AWS. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-hibernate-overview.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance... > When you initiate hibernation, the instance moves to the stopping state. Amazon EC2 signals the operating system to perform hibernation (suspend-to-disk). The hibernation freezes all of the processes, saves the contents of the RAM to the EBS root volume, and then performs a regular shutdown.
- nerpderp82 4y ago
  I believe under VMWare a suspension is transparent to the OS. The hypervisor just stops running the guest.
- tecleandor 4y ago
  I guess that time keeping could go crazy if you don't signal the OS to suspend or something like that. I remember some instances (years ago) where a VMWare live migration would make a VM clock jump or do something weird, and the guest OS would... "get scared".

outworlder 4y ago

We have been using this feature as 'beta' for a few years now with no issues. I've reported some bugs to GCP folks back when it was in an alpha state.

7373737373 4y ago

I wish more programming languages had this - would make it much easier to program simulations because agents could be written in the same language, instead of some other preemptible DSL, it'd also enable reproducible debugging and "system-jumping" processes if the suspended state could be serialized