6 ms·

Autoscaling, Welcome to Google Compute Engine

68 points by AndrewDucker 12y ago

pm90 12y ago
So, I'm actually intensely curios about how this thing works, even though its probably a trade secret or something. How do you prevent oscillations in resource allocations when the traffic is fluctuating? How do you know if e.g. an adversary is just DDoS'ing your application to make it more expensive to run your application? I guess there must be some kind of threshold that you can set? But then this would cut off the website for the real customers...
- bhouston 12y ago
  > an adversary is just DDoS'ing your application to make it more expensive to run your application? It isn't the compute engines job to determine if workload is malicious. That is a separate but real problem.
- ajessup 12y ago
  PM for GCE Autoscaling here. On protection from malicious traffic spikes - you can set a max-instances threshold to prevent over-spend which you can change at any time. Ultimately though, from an instance's perspective, there's not a lot of difference between a malicious DDoS and a genuine traffic spike. You can mitigate this (a) by setting low max-instances thresholds when you aren't expecting high traffic, and (b) upstream filtering or QOS management to filter our DDoS traffic from legitimate traffic.
  - _dark_matter_ 12y ago
    Do you have models for traffic pattern differences between spikes and DDoS attacks? It seems that a DDoS attack would have different patterns (quicker spike, no buildup, etc.). Just speculation here.
    - ajessup 12y ago
      Autoscaler will respond differently to different traffic patterns, but won't go as far as identifying a particular traffic pattern as malicious and refusing to grow resources to accommodate it (most architectures filter malicious requests upstream of the instance itself, eg. at the load balancer or via a proxy). Even if you could identify malicious usage, it's still a matter of policy on how you want to respond to it - given that if it can reach your VM it will likely be affecting legitimate traffic too.
  - jbkkd 12y ago
    Would it be possible in the future to set the thresholds based on time? Say I expect high traffic only during mornings, and not want to handle it during the evening if it occurs. Any chance this is in the making?
- rev_bird 12y ago
  > its probably a trade secret or something You can do auto-scaling without Amazon's secret sauce (or Google's), it just relies on triggers to spin up new boxes. I'm not very familiar with the Google setup, but you could use the Amazon API to pull metrics about boxes, then send a request to add another box to the auto-scaling group manually if you wanted. > How do you prevent oscillations in resource allocations when the traffic is fluctuating? Oscillation is kind of the point -- if you need x servers to serve y hits per hour, you'll have x servers when you need it. But when you're down to y/2 hits per hour, you don't need to be paying for x servers anymore. It just depends on load -- for example, you can set the group to add a box when average CPU rises above 70 percent, and then set it to remove a box when average CPU falls below 30 percent. > How do you know if e.g. an adversary is just DDoS'ing your application to make it more expensive to run your application? I guess there must be some kind of threshold that you can set? But then this would cut off the website for the real customers... This isn't a problem for auto-scaling, it's a problem for web stacks – the whole point of the "auto" part of it is that increase traffic means increased resources. Going counter to that gets complicated quickly. I'm sure there are folks who know more about DDoS mitigation than I do, but putting protection in front of your app with something like Akamai should help, as can keeping an eye out for bizarre spikes from certain IP ranges.
  - ajessup 12y ago
    It's worth noting that the GCE Autoscaler is configured quite differently to the AWS one. For the latter you define explicit scale conditions and triggers. On the former you define a utilisation metric and a target.
- cgio 12y ago
  Resource scaling is not that tough a problem computationally. The tough aspect of it is the practice of defining the driver of scale needs. Past that it is just calculating based on triggers a continuous or discrete scale. I have done it for FTEs in a huge mining operation and it worked like a charm (of course the requirement for scaling was not as real-time as in this case.) Use cases I had to also cater for in that case were: - reallocation from pools before scaling out, which is not very complex either, but it adds a bit of complexity, especially if you optimise for cost along the way. - linked scaling requirements, e.g. a team grows but there is a bottom and top limit to team size and when you get a new team you also need a supervisor On oscillations, you can weather them out by the way you treat your driver sampling window (static or variable) and a cost function. Normally oscillation is not bad for the resource allocation scenario, it is the intention of it, but for FTEs the cost can be more difficult to manage than for computational resources.
mmastrac 12y ago
This has been a long time coming. The autoscaler has been an interesting, and possible the most interesting, part of AppEngine and IMHO has recently reached the point (in the last year-and-a-half) where I'd say I'd want to rely on it for production. It wasn't always this way -- at one point we were having so many issues with AppEngine's autoscaling that we ended up in a meeting across from Ben Traynor and heard some of their engineers discussing post-mortems of scaling failures in gruesome detail. They've come a long way since then and AppEngine has really found its way from something of an internal science experiment to what I think is the best competitor for AWS' offerings. FWIW, the "backend" or "basic" scaling module that was available for both AppEngine and GCE was so poor and unreliable that it made it nearly impossible to scale anything in a way other than manual. I honestly hope they deprecate this and roll the few features into the autoscaler.
- erjiang 12y ago
  I'm confused - are you saying that scaling was horribly broken for App Engine, not just Compute Engine? Because it seems that it's much simpler to scale App Engine apps and App Engine has been around a long time.
  - mmastrac 12y ago
    Autoscaling was initially available for AppEngine only and the same (or similar) infrastructure is being made available to Compute Engine. Compute Engine had access to "basic scaling" which was so horribly broken and unreliable that you may as well just set up a static number of instances. The entire scaling infrastructure of AppEngine was horribly broken and unreliable up until a year-and-half or so ago, but they've put a lot of work into fixing that recently and it's in awesome shape now.
jedberg 12y ago
This is huge. When people ask me how someone could be a true competitor to AWS, my first is answer is usually "autoscaling". It's the feature that AWS has that no one else does (now did).
- mynameisvlad 12y ago
  Azure has it: http://azure.microsoft.com/en-us/documentation/articles/cloud-services-how-to-scale/ http://azure.microsoft.com/en-us/documentation/articles/clou... And based on the comments, has had it for 1-2 years now.
  - pm90 12y ago
    So does Rackspace, apparantly: http://www.rackspace.com/blog/easily-scale-your-cloud-with-rackspace-auto-scale/ http://www.rackspace.com/blog/easily-scale-your-cloud-with-r...
    - jedberg 12y ago
      It does, but they don't have nearly the capacity footprint that Amazon has.
  - jedberg 12y ago
    Azure has it and is a great platform for .Net workloads, but it's pretty terrible for Unix/Linux workloads.
- andrewguenther 12y ago
  After last week, that answer for me is now going to have to be "code deploy"
  - rev_bird 12y ago
    I'm pretty "meh" about CodeDeploy; though services are definitely what set AWS apart from competitors, moving code into production hasn't really been a pain point, especially with the "immutable infrastructure foreverrrr" paradigm they've been pushing and now seem to kind of be ignoring. Auto-scaling group launch configs make it easy, and Elastic Beanstalk works well for certain setups.
jakozaur 12y ago
Amazon Web Service plays the strategy "catch me if you can" (e.g. new services launched last week). Google is trying to catch up, if they want to beat AWS they would need quite a few differentiators (e.g. like buying Firebase).
- billjive 12y ago
  The problem with this statement is that Amazon is largely matching firebase functionality. They just announced streaming JSON out of dynamodb, and the new lambda functionality means you can trigger events on changes and push out updates. It wasn't explicitly stated but I viewed this as a firebase competitor.
TheMagicHorsey 12y ago
Is this the same sort of thing as AWS's Elastic Load Balancer service?
- andrewryno 12y ago
  It's like AWS EC2 Auto Scaling. GCP also has Load Balancing which complements the autoscaling feature, which is like AWS ELB.
- g9yuayon 12y ago
  No, it's not the same as ELB. It's similar to Amazon's auto scaling feature that comes with AWS Auto Scaling Groups. There's a key difference, though. AWS ASG is threshold based: users define actions when specified metrics exceed predefined threshold. In contrast, Google's autoscaling is goal oriented: users do not define action. They simplify specify desired values of specified metrics. Google takes care of scaling up or scaling down behind the scene. IMHO, Google's approach gives GCE more room for future optimization.
zeeshanm 12y ago
Would love to see autoscaling feature in Heroku. Wonder if it breaks their revenue model somehow to have something like this. May be it's more of a complex problem. As increasing dynos may not help for apps that are CPU bound.
- willcodeforfoo 12y ago
  I've wondered for a really long time myself https://news.ycombinator.com/item?id=7104653 https://news.ycombinator.com/item?id=7104653
pdkl95 12y ago
So the "Google Grid"[1] is finally complete? [1] http://en.wikipedia.org/wiki/EPIC_2014 http://en.wikipedia.org/wiki/EPIC_2014

68 points by AndrewDucker 12y ago

pm90 12y ago

So, I'm actually intensely curios about how this thing works, even though its probably a trade secret or something. How do you prevent oscillations in resource allocations when the traffic is fluctuating? How do you know if e.g. an adversary is just DDoS'ing your application to make it more expensive to run your application? I guess there must be some kind of threshold that you can set? But then this would cut off the website for the real customers...

bhouston 12y ago
> an adversary is just DDoS'ing your application to make it more expensive to run your application? It isn't the compute engines job to determine if workload is malicious. That is a separate but real problem.
ajessup 12y ago
PM for GCE Autoscaling here. On protection from malicious traffic spikes - you can set a max-instances threshold to prevent over-spend which you can change at any time. Ultimately though, from an instance's perspective, there's not a lot of difference between a malicious DDoS and a genuine traffic spike. You can mitigate this (a) by setting low max-instances thresholds when you aren't expecting high traffic, and (b) upstream filtering or QOS management to filter our DDoS traffic from legitimate traffic.
- _dark_matter_ 12y ago
  Do you have models for traffic pattern differences between spikes and DDoS attacks? It seems that a DDoS attack would have different patterns (quicker spike, no buildup, etc.). Just speculation here.
  - ajessup 12y ago
    Autoscaler will respond differently to different traffic patterns, but won't go as far as identifying a particular traffic pattern as malicious and refusing to grow resources to accommodate it (most architectures filter malicious requests upstream of the instance itself, eg. at the load balancer or via a proxy). Even if you could identify malicious usage, it's still a matter of policy on how you want to respond to it - given that if it can reach your VM it will likely be affecting legitimate traffic too.
- jbkkd 12y ago
  Would it be possible in the future to set the thresholds based on time? Say I expect high traffic only during mornings, and not want to handle it during the evening if it occurs. Any chance this is in the making?
rev_bird 12y ago
> its probably a trade secret or something You can do auto-scaling without Amazon's secret sauce (or Google's), it just relies on triggers to spin up new boxes. I'm not very familiar with the Google setup, but you could use the Amazon API to pull metrics about boxes, then send a request to add another box to the auto-scaling group manually if you wanted. > How do you prevent oscillations in resource allocations when the traffic is fluctuating? Oscillation is kind of the point -- if you need x servers to serve y hits per hour, you'll have x servers when you need it. But when you're down to y/2 hits per hour, you don't need to be paying for x servers anymore. It just depends on load -- for example, you can set the group to add a box when average CPU rises above 70 percent, and then set it to remove a box when average CPU falls below 30 percent. > How do you know if e.g. an adversary is just DDoS'ing your application to make it more expensive to run your application? I guess there must be some kind of threshold that you can set? But then this would cut off the website for the real customers... This isn't a problem for auto-scaling, it's a problem for web stacks – the whole point of the "auto" part of it is that increase traffic means increased resources. Going counter to that gets complicated quickly. I'm sure there are folks who know more about DDoS mitigation than I do, but putting protection in front of your app with something like Akamai should help, as can keeping an eye out for bizarre spikes from certain IP ranges.
- ajessup 12y ago
  It's worth noting that the GCE Autoscaler is configured quite differently to the AWS one. For the latter you define explicit scale conditions and triggers. On the former you define a utilisation metric and a target.
cgio 12y ago
Resource scaling is not that tough a problem computationally. The tough aspect of it is the practice of defining the driver of scale needs. Past that it is just calculating based on triggers a continuous or discrete scale. I have done it for FTEs in a huge mining operation and it worked like a charm (of course the requirement for scaling was not as real-time as in this case.) Use cases I had to also cater for in that case were: - reallocation from pools before scaling out, which is not very complex either, but it adds a bit of complexity, especially if you optimise for cost along the way. - linked scaling requirements, e.g. a team grows but there is a bottom and top limit to team size and when you get a new team you also need a supervisor On oscillations, you can weather them out by the way you treat your driver sampling window (static or variable) and a cost function. Normally oscillation is not bad for the resource allocation scenario, it is the intention of it, but for FTEs the cost can be more difficult to manage than for computational resources.

mmastrac 12y ago

This has been a long time coming. The autoscaler has been an interesting, and possible the most interesting, part of AppEngine and IMHO has recently reached the point (in the last year-and-a-half) where I'd say I'd want to rely on it for production. It wasn't always this way -- at one point we were having so many issues with AppEngine's autoscaling that we ended up in a meeting across from Ben Traynor and heard some of their engineers discussing post-mortems of scaling failures in gruesome detail. They've come a long way since then and AppEngine has really found its way from something of an internal science experiment to what I think is the best competitor for AWS' offerings. FWIW, the "backend" or "basic" scaling module that was available for both AppEngine and GCE was so poor and unreliable that it made it nearly impossible to scale anything in a way other than manual. I honestly hope they deprecate this and roll the few features into the autoscaler.

erjiang 12y ago
I'm confused - are you saying that scaling was horribly broken for App Engine, not just Compute Engine? Because it seems that it's much simpler to scale App Engine apps and App Engine has been around a long time.
- mmastrac 12y ago
  Autoscaling was initially available for AppEngine only and the same (or similar) infrastructure is being made available to Compute Engine. Compute Engine had access to "basic scaling" which was so horribly broken and unreliable that you may as well just set up a static number of instances. The entire scaling infrastructure of AppEngine was horribly broken and unreliable up until a year-and-half or so ago, but they've put a lot of work into fixing that recently and it's in awesome shape now.

jedberg 12y ago

This is huge. When people ask me how someone could be a true competitor to AWS, my first is answer is usually "autoscaling". It's the feature that AWS has that no one else does (now did).

mynameisvlad 12y ago
Azure has it: http://azure.microsoft.com/en-us/documentation/articles/cloud-services-how-to-scale/ http://azure.microsoft.com/en-us/documentation/articles/clou... And based on the comments, has had it for 1-2 years now.
- pm90 12y ago
  So does Rackspace, apparantly: http://www.rackspace.com/blog/easily-scale-your-cloud-with-rackspace-auto-scale/ http://www.rackspace.com/blog/easily-scale-your-cloud-with-r...
  - jedberg 12y ago
    It does, but they don't have nearly the capacity footprint that Amazon has.
- jedberg 12y ago
  Azure has it and is a great platform for .Net workloads, but it's pretty terrible for Unix/Linux workloads.
andrewguenther 12y ago
After last week, that answer for me is now going to have to be "code deploy"
- rev_bird 12y ago
  I'm pretty "meh" about CodeDeploy; though services are definitely what set AWS apart from competitors, moving code into production hasn't really been a pain point, especially with the "immutable infrastructure foreverrrr" paradigm they've been pushing and now seem to kind of be ignoring. Auto-scaling group launch configs make it easy, and Elastic Beanstalk works well for certain setups.

jakozaur 12y ago

Amazon Web Service plays the strategy "catch me if you can" (e.g. new services launched last week). Google is trying to catch up, if they want to beat AWS they would need quite a few differentiators (e.g. like buying Firebase).

billjive 12y ago
The problem with this statement is that Amazon is largely matching firebase functionality. They just announced streaming JSON out of dynamodb, and the new lambda functionality means you can trigger events on changes and push out updates. It wasn't explicitly stated but I viewed this as a firebase competitor.

TheMagicHorsey 12y ago

Is this the same sort of thing as AWS's Elastic Load Balancer service?

andrewryno 12y ago
It's like AWS EC2 Auto Scaling. GCP also has Load Balancing which complements the autoscaling feature, which is like AWS ELB.
g9yuayon 12y ago
No, it's not the same as ELB. It's similar to Amazon's auto scaling feature that comes with AWS Auto Scaling Groups. There's a key difference, though. AWS ASG is threshold based: users define actions when specified metrics exceed predefined threshold. In contrast, Google's autoscaling is goal oriented: users do not define action. They simplify specify desired values of specified metrics. Google takes care of scaling up or scaling down behind the scene. IMHO, Google's approach gives GCE more room for future optimization.

zeeshanm 12y ago

Would love to see autoscaling feature in Heroku. Wonder if it breaks their revenue model somehow to have something like this. May be it's more of a complex problem. As increasing dynos may not help for apps that are CPU bound.

willcodeforfoo 12y ago
I've wondered for a really long time myself https://news.ycombinator.com/item?id=7104653 https://news.ycombinator.com/item?id=7104653

pdkl95 12y ago

So the "Google Grid"[1] is finally complete? [1] http://en.wikipedia.org/wiki/EPIC_2014 http://en.wikipedia.org/wiki/EPIC_2014