10 ms·
Introducing Preemptible VMs
- gnur 11y agoI wonder how big the chance is that the machine is turned off in the first hour. I use Google Compute for some personal project and the typical run time for a VM is about 10 - 20 minutes. If the vm has 90% chance to survive the first hour, it could be worth the trouble to make my process more fault tolerable.
- boulos 11y agoWe're not ready to put hard numbers on it. As our docs (https://cloud.google.com/compute/docs/instances/preemptible https://cloud.google.com/compute/docs/instances/preemptible) say: The probability that Compute Engine will terminate a preemptible instance for a system event is generally low, but may vary from day to day and from zone to zone depending on current conditions. Give it a shot in us-central1-a and let us know how it goes! [Edit to undo my quote text (it wrapped poorly)]
- pacala 11y agoI couldn't figure out from the docs, can one restart a preempted preemptible instance, or needs to start a different one? Would it be possible to restart in non-preemptible mode, so the job completes but at a higher price? We still want to complete our workloads, one way or another :)
- boulos 11y agoYes you can start a VM back up via gcloud: gcloud compute instances start instance-name Note of course that if you got preempted because we needed the capacity back for regular VMs (as opposed to say a maintenance event) you may not be able to start a new Preemptible VM in that zone.
- ikeboy 11y agoHow does their pricing compare to ec2 spot instances?
- boulos 11y agoYou can find a list of our prices here: https://cloud.google.com/compute/#pricing https://cloud.google.com/compute/#pricing . Preemptible VMs have flat per-machine type pricing compared to EC2's bidding-based pricing. For example, a Preemptible n1-standard-1 is 1.5 cents per hour in the US and a preemptible n1-standard-4 is 6.0 cents per hour.
- miles932 11y agoWell, spot varies. Sometimes it's low, but the price varies based on demand. Using prices as of right now for example, Currently in California, c3.8xl instances are quite popular, running $1.68 an hour on an instance that's normally 1.912 an hour, roughly a 13% discount. 70% strikes me as better than 13%, especially when it's predictable. (and 70% below an already substantially lower hourly price...) [1] http://storage.googleapis.com/pricingcomparator.appspot.com/spot.png http://storage.googleapis.com/pricingcomparator.appspot.com/... [2] http://storage.googleapis.com/pricingcomparator.appspot.com/on-demand.png http://storage.googleapis.com/pricingcomparator.appspot.com/... disclaimer: I work at Google on Google Cloud Platform
- vosper 11y agoThe long-run price for a c3.8xlarge in California is somewhere between $0.40 and $0.50, though you're correct that right now they're $1.68. California tends to be more expensive than other AWS regions, though (I can't remember the reason - perhaps just availability?). If you're in us-east-1 the price sticks around $0.20 per hour, and eu-west-1 is rock-solid at $0.32 per hour. If you have work that can be done on spots it can be a good idea to make it region-agnostic so that you can take advantage of better prices for different instances in different regions.
- vgt 11y agoThe idea with Google is that you don't need to worry about which instances you run, at which price, and in which region. For example, with Google you don't need to get a specialized instance type to use it's monster-fast Local SSD - just use the same instance types. This alone should simplify use of Preemptible VMs.
- deleted 11y ago[deleted]
- TallGuyShort 11y agoIt isn't fundamentally different from EC2 spot instances.
- raverbashing 11y agoBeyond the fixed price, no? Also, do you get the 30s grace period on spot instance shutdown?
- acdha 11y agoAs of January, yes: https://aws.amazon.com/blogs/aws/new-ec2-spot-instance-termination-notices/ https://aws.amazon.com/blogs/aws/new-ec2-spot-instance-termi...
- jread 11y agoAnd 24 hour run limit
- TallGuyShort 11y agoRight, but the idea of getting kicked out once someone's willing to pay more is not a new one.
- hillsarealiv3 11y agoEC2 spot gives you 2 minutes, maybe even enough time to launch a replacement.
- boulos 11y agoPreemptible VMs definitely aren't for everything. We just lowered our pricing on regular VMs (https://news.ycombinator.com/item?id=9564136 https://news.ycombinator.com/item?id=9564136) as well, so if your workload isn't fault tolerant I'm sure our full price hotel is just fine ;).
- vosper 11y ago
- vosper 11y ago@boulos, since it looks like you're involved with this: The link to the product pricing [1] goes to a generic landing page for Google services, which is a little confusing. I think you should link directly to the instance pricing [2]. [1] https://cloud.google.com/pricing/#compute-engine https://cloud.google.com/pricing/#compute-engine [2] https://cloud.google.com/compute/pricing https://cloud.google.com/compute/pricing
- boulos 11y agoYeah, as part of our larger pricing announcement we're making sure people go to /pricing so they get a bigger picture before diving straight to the table ;) [Edit: Point taken though]
- branola 11y agoAny developer who was burned by App Engine pricing in the past is likely to take the lure of a "70% off" discount with a grain of salt because of the much higher cost of incorrectly relying on Google's pricing pledges. In case anyone forgets, the pricing changes on Google App Engine caused many developers to abandon apps they had developed on the platform because of hundreds of percent increases in pricing: https://groups.google.com/forum/#!topic/google-appengine/rSh8mgw2f3c https://groups.google.com/forum/#!topic/google-appengine/rSh...
- spankalee 11y agoApp Engine was in preview when its price changed.
- branola 11y agoIt was in "preview" for 3 years. During those 3 years no warning was given that the pricing changes were going to be of such a magnitude. As a result people had sufficient time and not enough warning to build entire businesses on an infrastructure that they later had to abandon. That's unforgivable.
- Alupis 11y ago> build entire businesses on an infrastructure that they later had to abandon. Frankly, if your business is built entirely around a single 3rd party provider, and you are totally incapable of pivoting cloud providers at short notice... well then you are "doing it wrong".
- discardorama 11y agoLike Netflix? They seem to be doing alright.
- Alupis 11y ago> Like Netflix? They seem to be doing alright. Could you elaborate a little? Netflix doesn't rely on a single provider for hardware, data centers, nor bandwidth. Sure they use EC2 for some things, but also have a great deal of hardware in data centers throughout the country, as well as custom hardware in ISP data centers, etc. I'd be shocked if there didn't use some of the other cloud provider offering too. Netflix has no single point of failure. Building your business around a single cloud provider creates a single point of failure.
- andybak 11y agoAny reason you couldn't use these for conventional web stuff? 30 seconds could easily be long enough to bring up another instance and sync data if needed.
- CydeWeys 11y agoBy "conventional web stuff", do you mean serving a website or similar? Yeah, I don't think it makes sense to host a website on a service that is preemptible. You need your webserver available to respond to requests 24/7. It can't just randomly go down. Pre-emptible instances are more suited for batch processing and big data computation runs.
- comex 11y agoIf you have a website that can be distributed among arbitrarily many frontend servers, it could make sense to put them on preemptible instances, with the database on a regular instance. (In the unlikely case no preemptible instances are available, you could always automatically switch the frontend servers to regular ones.) However, I think this could only possibly be efficient pricing-wise if your traffic was extremely bursty.
- spankalee 11y agoNo reason at all. If you have enough preemptible VMs, and they spin up fast enough, then when one goes down, just spin up another. If for some reason GCE's overall preemptible spare capacity drops and you can't get a new one and it's important to you, then spin up a regular VM.
- ErikRogneby 11y agoI would say it depends on the size of your pool. 30 seconds _is not_ long enough to bring up another instance.
- hillsarealiv3 11y agoEC2 spot instances give 2 minutes notice...
- jxf 11y agoThis is a great announcement. Preemptible VMs are a great strategy if: * you don't care too much about when a workload finishes * it can be chunked into small units of interruptible work * the cost of resuming interrupted work is zero or small Examples of workloads that might look like this: * geocoding large batches of addresses * map-reducing large event streams * content scraping/crawling Notice that you don't need to run all of your cluster as preemptible if you're using bdutil [0], too -- some of the workload can be preemptible, and some not. So you can guarantee a minimum processing throughput, but get extra throughput for very cheap. I think that's a great way to do it, though I wish it was generalized for all kinds of clusters, not just for Hadoop ones. [0] https://github.com/GoogleCloudPlatform/bdutil https://github.com/GoogleCloudPlatform/bdutil
- boulos 11y agoYeah, I added it to bdutil since it was clearly useful and yet well understood that it would do the right thing. I'd love to see more cluster and machine management tools deal with "machines can come and go". The guys on the go team are using it for their buildbots (http://build.golang.org http://build.golang.org) which was already prepared for the box to die.
- mark_l_watson 11y agoNice! This ties in with the discussion yesterday on HN about data center utilization. Good for saving money and the end result must be lower global energy use if this also catches on with other providers (AWS already offers something similar). When I worked at Google in 2013 part of my job was running very large calculations that were often pre-empted. I usually ran at the lowest priority and was in effect using spare capacity. No hassles, assuming that getting runs completed was not too time sensitive.
- GreyTheory 11y agoCould you point me at the datacenter utilization article? I can't seem to find it, and it sounds interesting
- 4k 11y agoI might be missing something important here. But according to their pricing page[1], a preemtible VM with 30GB memory costs $86.4 a month. Why would someone go for this when cheap dedicated host providers like hetzner etc offer powerful dedicated servers with 64GB memory and multicore server grade CPUs? The comparison only gets worse taking into account that Google's offering is preemptible and can shutdown and come up as they wish. [1]: https://cloud.google.com/compute/#pricing https://cloud.google.com/compute/#pricing (0.12x24x30)
- CydeWeys 11y agoIt's about scaling. Google's offering is a cloud service and Hetzner's is not. You're always paying the same amount per month with Hetzner, whereas you'd only pay that much with Google if you have consistent full utilization (which seems rather unlikely; most workloads do not look like this). Secondly, if you create something that becomes popular, you can scale it up on GCE a lot more easily than you can with a dedicated hosting provider.
- rwmj 11y agoIt's like asking why do people use Serviced Offices which are usually 10x the price of a monthly rented office. Answers: because they only need the office / VM for a few hours. Because they can immediately get an office / VM of any size they need. Because they can walk away from the office / VM when they don't need it any longer. It's certainly not for everyone, but Regus seem to be doing ok.
- vidarh 11y agoAnd for those who truly only need it now and again, it's great. But time and time again I see infrastructure where people pay for these services for large amount of instances that are running continuously, blindly assuming that it's cheap because it's cloud. There's a bizarre level of price-blindness amongst certain subset of customers of Google Cloud and AWS that I've never seen anywhere else.
- noelwelsh 11y agoAPIs, instant access, massive scale, no setup fees, and value-added services like DBs and message queues are all reasons to use a cloud provider.
- tw04 11y agoSo... this is essentially first come, first served based on their description. And they say the preemptibles come from a smaller pool of resources. Seems odd to me they wouldn't have a number somewhere of how many are available and how many are in use given that it's a finite resource. Tough to build a business model around a resource you can't even determine the availability of.
- nightcracker 11y ago> Though to build a business model around a resource you can't even determine the availability of. That's exactly why it's cheap: you're trading price for reliability.
- tw04 11y agoreliability != availability. I can build a business model around an unreliable, but available resource. I can't build a business model around an unavailable resource. They're two very different things, and the distinction is important.
- nl 11y agoThis basically for batch-processing. That means that the lack of availability isn't such an issue because this lets you trade speed-of-processing for cost. Say you have a few hundred terabytes of images to process. You can prioritize images by pushing them to the head of the queue, but you don't really care how long the complete batch takes. If you are happy to wait for your job to complete you pay less. Otherwise, pay more and guarantee completion.
- boulos 11y agoYou're right that it's first come first served, but the 24 hour time limit combined with natural churn means you aren't likely to be "locked out" for long periods of time.
- h43k3r 11y agoNow I am waiting to see a similar thing in Azure. Preemptible VMs will prove to be very useful in fault tolerant Distributed Networks. One more usecase that I can think off is Load Testing on a large scale ( in a distributed way ).
- burner21926 11y agoThis is awesome. Thank you Google for another great addition to hassle-free cloud computing.
- SEJeff 11y agoAnd this is where a tool like mesos + aurora/marathon really really shines. The same can be said for the likes of kubernetes.
- deadbunny 11y agoCould certainly be interesting for cheaply expanding Mesos slaves to cope with a workload spike.
- boulos 11y agoAs we put in the docs, to try this out: gcloud compute instances create my-vm --preemptible --zone=us-central1-c or just set the preemptible bool to true via the API (it's under scheduling). When you want to test how your system behaves on preemption, you can just do: gcloud compute instances stop my-vm --zone=us-central1-c which will give you the same 30 second timeout as when you're preempted. Most OSes have a fairly standard set of things they do on shutdown that will at least send all your running processes a signal (via kill), but if you need to add your own you can inject it via the new shutdown script support (https://cloud.google.com/compute/docs/shutdownscript https://cloud.google.com/compute/docs/shutdownscript). We tried to cover this in the docs (https://cloud.google.com/compute/docs/instances/preemptible https://cloud.google.com/compute/docs/instances/preemptible) any feedback on that would be welcome!
- Tegran 11y agoThe 1970s just called. They want their batch processing back. :-)
- bitJericho 11y agoAt a penny per core per instance hour the cost is about 7 dollars a month. I can get a normal dedicated VM for cheaper... What's the benefit in this?
- jpatokal 11y ago> What's the benefit in this? You're not buying a dedicated instance for a month, you're renting it as long as you need it. If you only need a core for an hour, your bill will be $0.01.