5 ms·

If you are running Kubernetes http://valence.net http://valence.net might be interesting

by jmacd 7y ago

jacques_chester 7y ago
Autoscaling is both simpler than this and more complex than this. For predictive autoscaling, boring old-fashioned forecasting techniques appear to work fine and are very fast and very cheap. For reactive autoscaling, boring old-fashioned control loops appear to work fine and are very fast and very cheap. There's still no substitute for understanding your application's design. If you tried to design and build a factory the way folks propose to rely on autoscalers and ML ("we'll ignore what's in the building and rely on a thin, unintelligible gas of floats!!"), you would be fired.
- bayareanative 7y ago
  Yeap. Autoscaling strategies are often traditionally hand-coded to depend on the costs associated of not completing transactions or site going down from a business impact analysis (BIA). To add AI/ML would need a number of business rules, metrics, automation parameter limitation and (a) fitness function(s) in order to use it. Examples of hard-coded autoscaling strategies: Shopping cart or ad delivery network -> anticipate load with extra capacity by scaling up instances before they're needed and kept until they aren't beneficial. Neighborhood social network: - scale +1 instances after avg latency > X0 ms for Y0 minutes - scale -1 after avg latency < X1 ms for Y1 minutes
  - mlabouardy 7y ago
    autoscaling strategies can help you to reduce your cost, however people often forget that small things like unused disks, snapshots, nat gateway can cost you a fortune
  - jacques_chester 7y ago
    > Autoscaling strategies are often traditionally hand-coded to depend on the costs associated of not completing transactions or site going down from a business impact analysis (BIA). Adding on this, my current thinking is that it would be possible to better surface this tradeoff by modeling autoscaling as an inventory problem. There's a stockout cost and a holding cost; for a given probability of hitting a cold start and for a given cold start lag, it should be possible to compute the optimal instances on-hand. Then there are, as you point out, many special business rules. Where I work we have a pool system for testing environments. It works reasonably well until a particular team, sitting upstream of approximately half the company, makes a release. Then suddenly everyone's pipelines kick off simultaneously. So now they have a system which watches for signs of impending release and pre-emptively scales up. Autoscaling should not be your first line or final line of defence. It is a powerful tool, not a magic wand.
edaemon 7y ago
https://valence.net/ https://valence.net/ works, but it looks like HTTP gets the default nginx page.

If you are running Kubernetes http://valence.net http://valence.net might be interesting

by jmacd 7y ago

If you are running Kubernetes http://valence.net http://valence.net might be interesting

Autoscaling is both simpler than this and more complex than this. For predictive autoscaling, boring old-fashioned forecasting techniques appear to work fine and are very fast and very cheap. For reactive autoscaling, boring old-fashioned control loops appear to work fine and are very fast and very cheap. There's still no substitute for understanding your application's design. If you tried to design and build a factory the way folks propose to rely on autoscalers and ML ("we'll ignore what's in the building and rely on a thin, unintelligible gas of floats!!"), you would be fired.

bayareanative 7y ago
Yeap. Autoscaling strategies are often traditionally hand-coded to depend on the costs associated of not completing transactions or site going down from a business impact analysis (BIA). To add AI/ML would need a number of business rules, metrics, automation parameter limitation and (a) fitness function(s) in order to use it. Examples of hard-coded autoscaling strategies: Shopping cart or ad delivery network -> anticipate load with extra capacity by scaling up instances before they're needed and kept until they aren't beneficial. Neighborhood social network: - scale +1 instances after avg latency > X0 ms for Y0 minutes - scale -1 after avg latency < X1 ms for Y1 minutes
- mlabouardy 7y ago
  autoscaling strategies can help you to reduce your cost, however people often forget that small things like unused disks, snapshots, nat gateway can cost you a fortune
- jacques_chester 7y ago
  > Autoscaling strategies are often traditionally hand-coded to depend on the costs associated of not completing transactions or site going down from a business impact analysis (BIA). Adding on this, my current thinking is that it would be possible to better surface this tradeoff by modeling autoscaling as an inventory problem. There's a stockout cost and a holding cost; for a given probability of hitting a cold start and for a given cold start lag, it should be possible to compute the optimal instances on-hand. Then there are, as you point out, many special business rules. Where I work we have a pool system for testing environments. It works reasonably well until a particular team, sitting upstream of approximately half the company, makes a release. Then suddenly everyone's pipelines kick off simultaneously. So now they have a system which watches for signs of impending release and pre-emptively scales up. Autoscaling should not be your first line or final line of defence. It is a powerful tool, not a magic wand.

edaemon 7y ago

https://valence.net/ https://valence.net/ works, but it looks like HTTP gets the default nginx page.