6 ms·

Compute Engine Predictive Autoscaling

53 points by tzury 5y ago

davidjytang 5y ago
My experience with GCE's autoscaling has been that the up-scaling is much more aggressive than anticipated. I see with this predictive autoscaling, up-scaling would be even more aggressive.
- lofties 5y ago
  And the downscaling in my experience takes too long. Sometimes it feels it's done this way on purpose to rake in a few extra hours of compute pennies. It starts adding up... imagine if you have a million customers and you can suddenly charge them an extra few dollars per month.
  - klenwell 5y ago
    I have an app that has been running on the old Python 2.7 App Engine platform for a few years. It gets very light traffic and usually costs $1 or $2 a month, mainly for datastore read ops. Suddenly this month there was an additional $11.65 charge for "frontend instances". I've searched the dashboard, documentation, and my configuration files trying to figure out what this means. I still have no idea where it came from or how to stop it, short of turning off the app.
    - ahmedtd 5y ago
      If you search for "frontend" on the App Engine Standard Pricing Page [1], it seems like your usage might be exceeding the free tier for frontend instances. Did the load on your app suddenly increase? [1] https://cloud.google.com/appengine/pricing https://cloud.google.com/appengine/pricing
      - klenwell 5y ago
        Yeah, that makes sense. A load increase that tripped the autoscaler to add a second instance that put me over free limits for the first time. I think I figured out how to stop it from doing this: https://old.reddit.com/r/AppEngine/comments/od11sg https://old.reddit.com/r/AppEngine/comments/od11sg Thanks for the link.
- otterley 5y ago
  At least with AWS, auto scaling is biased towards aggressive scale-outs and conservative scale-ins, under the general principle that customers would prefer to err on the side of providing service and meeting their SLAs over conserving every last dollar of spend. It makes sense that GCE would adopt the same attitude.
Frajedo 5y ago
I am glad Google is finally trying to catch up with the basic features of AWS infinidash. They have still got a lot to do!
- jiggawatts 5y ago
  What does it say about AWS branding that even after googling the term and reading the top three hits, I can't tell if "AWS infinidash" is a joke or a real product?
  - erwinh 5y ago
    To prevent others from losing time googling, its a joke ;) https://twitter.com/jna_sh/status/1410178986978775040?s=20 https://twitter.com/jna_sh/status/1410178986978775040?s=20 Did produce some quality comedy material! => https://twitter.com/TartanLlama/status/1410959645238308866?s=20 https://twitter.com/TartanLlama/status/1410959645238308866?s...
sendtopms 5y ago
scalling by adding CPU Cores and RAM would also be useful for certain workloads, I did not see this option.
- corty 5y ago
  Maybe that is step 2. I agree that this would be hugely beneficial e.g. for databases where a low number of very fat replicas is preferable.
amelius 5y ago
Does it monitor slashdot to anticipate the slashdot effect?
dijit 5y ago
Back when I was working at Massive Entertainment (part of Ubisoft) we actually implemented a predictive autoscaler on top of SaltStacks REST API -- in C++ of course for The Division 2[0] This would have been quite handy back then, it seems to operate on almost exactly the same principles. The hardest parts were the fact that it was a 10-20 minute endevour to create a new gameserver instance, and predictions get more difficult the further into the future they are. We didn't use any fancy AI stuff, just linear regression with smoothing across the history. We preferred to overshoot, but the predictions had a tendency to spike sometimes with relatively normal trends. You don't want to start creating a machine with a 20 minute lead time and abort it. It's a wasted hour of compute time. (and an hour is the minimum windows licensing cost). Windows licensing was 50% of our costs on GCP. [0]: https://cloud.google.com/customers/massive-entertainment https://cloud.google.com/customers/massive-entertainment
- ksec 5y ago
  Sorry if this is a dumb question. >a new gameserver instance.........Windows licensing was 50% of our costs on GCP. Are there any reason why the gameServer part is on Windows? And if this is a norm in Games industry. I always assumed the Server part would be a Linux stack.
  - dijit 5y ago
    Everyone has the same question, so it's not a stupid one for sure. I answered this a bit more in depth here: https://www.gcppodcast.com/post/episode-265-sharkmob-games-with-jan-harasym/ https://www.gcppodcast.com/post/episode-265-sharkmob-games-w... but tl;dr: When you tell developers that they can't run VMs and new hardware takes 6-12 months and a lot of bureaucracy to acquire, you make it work on your machine. And, of course, Gamedevs understand win32 API very well. And, making high performance networked AsyncIO is a bit nicer on Windows (IOCP vs EPOLL).
shaicoleman 5y ago
In AWS, the equivalent is called Predictive Scaling: https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-predictive-scaling.html https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-au...

53 points by tzury 5y ago

My experience with GCE's autoscaling has been that the up-scaling is much more aggressive than anticipated. I see with this predictive autoscaling, up-scaling would be even more aggressive.

lofties 5y ago
And the downscaling in my experience takes too long. Sometimes it feels it's done this way on purpose to rake in a few extra hours of compute pennies. It starts adding up... imagine if you have a million customers and you can suddenly charge them an extra few dollars per month.
- klenwell 5y ago
  I have an app that has been running on the old Python 2.7 App Engine platform for a few years. It gets very light traffic and usually costs $1 or $2 a month, mainly for datastore read ops. Suddenly this month there was an additional $11.65 charge for "frontend instances". I've searched the dashboard, documentation, and my configuration files trying to figure out what this means. I still have no idea where it came from or how to stop it, short of turning off the app.
  - ahmedtd 5y ago
    If you search for "frontend" on the App Engine Standard Pricing Page [1], it seems like your usage might be exceeding the free tier for frontend instances. Did the load on your app suddenly increase? [1] https://cloud.google.com/appengine/pricing https://cloud.google.com/appengine/pricing
    - klenwell 5y ago
      Yeah, that makes sense. A load increase that tripped the autoscaler to add a second instance that put me over free limits for the first time. I think I figured out how to stop it from doing this: https://old.reddit.com/r/AppEngine/comments/od11sg https://old.reddit.com/r/AppEngine/comments/od11sg Thanks for the link.
otterley 5y ago
At least with AWS, auto scaling is biased towards aggressive scale-outs and conservative scale-ins, under the general principle that customers would prefer to err on the side of providing service and meeting their SLAs over conserving every last dollar of spend. It makes sense that GCE would adopt the same attitude.

Frajedo 5y ago

I am glad Google is finally trying to catch up with the basic features of AWS infinidash. They have still got a lot to do!

jiggawatts 5y ago
What does it say about AWS branding that even after googling the term and reading the top three hits, I can't tell if "AWS infinidash" is a joke or a real product?
- erwinh 5y ago
  To prevent others from losing time googling, its a joke ;) https://twitter.com/jna_sh/status/1410178986978775040?s=20 https://twitter.com/jna_sh/status/1410178986978775040?s=20 Did produce some quality comedy material! => https://twitter.com/TartanLlama/status/1410959645238308866?s=20 https://twitter.com/TartanLlama/status/1410959645238308866?s...

sendtopms 5y ago

scalling by adding CPU Cores and RAM would also be useful for certain workloads, I did not see this option.

corty 5y ago
Maybe that is step 2. I agree that this would be hugely beneficial e.g. for databases where a low number of very fat replicas is preferable.

amelius 5y ago

Does it monitor slashdot to anticipate the slashdot effect?

dijit 5y ago

Back when I was working at Massive Entertainment (part of Ubisoft) we actually implemented a predictive autoscaler on top of SaltStacks REST API -- in C++ of course for The Division 2[0] This would have been quite handy back then, it seems to operate on almost exactly the same principles. The hardest parts were the fact that it was a 10-20 minute endevour to create a new gameserver instance, and predictions get more difficult the further into the future they are. We didn't use any fancy AI stuff, just linear regression with smoothing across the history. We preferred to overshoot, but the predictions had a tendency to spike sometimes with relatively normal trends. You don't want to start creating a machine with a 20 minute lead time and abort it. It's a wasted hour of compute time. (and an hour is the minimum windows licensing cost). Windows licensing was 50% of our costs on GCP. [0]: https://cloud.google.com/customers/massive-entertainment https://cloud.google.com/customers/massive-entertainment

ksec 5y ago
Sorry if this is a dumb question. >a new gameserver instance.........Windows licensing was 50% of our costs on GCP. Are there any reason why the gameServer part is on Windows? And if this is a norm in Games industry. I always assumed the Server part would be a Linux stack.
- dijit 5y ago
  Everyone has the same question, so it's not a stupid one for sure. I answered this a bit more in depth here: https://www.gcppodcast.com/post/episode-265-sharkmob-games-with-jan-harasym/ https://www.gcppodcast.com/post/episode-265-sharkmob-games-w... but tl;dr: When you tell developers that they can't run VMs and new hardware takes 6-12 months and a lot of bureaucracy to acquire, you make it work on your machine. And, of course, Gamedevs understand win32 API very well. And, making high performance networked AsyncIO is a bit nicer on Windows (IOCP vs EPOLL).

shaicoleman 5y ago

In AWS, the equivalent is called Predictive Scaling: https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-predictive-scaling.html https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-au...