4 ms·

I disagree. Autoscaling sounds great, but in practice it notices that there is a problem after you already have once, and by the time you've scaled up, there i

by btilly 7y ago

I disagree. Autoscaling sounds great, but in practice it notices that there is a problem after you already have once, and by the time you've scaled up, there is no problem any more. Which is the worst of all worlds. For example in this case you'd realize there is a problem at the top of the hour. And 2 minutes later you'd have a bunch of autoscaled instances up, all wondering what the fuss was about. So they scale down again, but then at the top of the next hour the same thing happens again with the same result. The same thing happens with ad servers. An ad buy goes in for 30 seconds. When it hits, you have a firehose. That then shuts off. In fact this problem is so common that I recommend against autoscaling unless you have specific reason to believe that it will work for you.

auslander 7y ago
> For example .. In fact this problem is so common .. So whats the problem here? Assuming there are no misconfigurations anymore, and just spiking loads.
- btilly 7y ago
  The problem is that people try to use autoscaling to handle a spiking load, and it fails abysmally. In my opinion for those with a spiking load profile, autoscaling should be considered harmful.
hinkley 7y ago
And god help you if that starts your autoscaler to flapping. Too much CPU load, start some servers. Too little, kill some servers. Oops now CPU is too high, bring them back. No, it's too low... It reminds me of the old UI bug where you get an off-by-one error for scrollbar logic and so it appears and disappears in a cycle until you resize. Edit: I think we want something akin to a learning thermostat, but for servers. Figure out that I get a CPU spike at 5:20 on week days and spin some servers up at 5:10. Then spin em down at 8:00 when EST starts to go to bed.
- auslander 7y ago
  > .. Oops now CPU is too high, bring them back. No, it's too low... You just described closed-loop control system oscillation. Cause is wrong gain and/or delay parameters. AWS ASG has some knobs for tuning like Cooldown, MaxSize. What you described above is most probably long boot time (delay) problem. Service should be ready in less than 30 secs after boot. To get there, one should bake AMIs, not install afresh all the stuff on boot.
  - hinkley 7y ago
    You can get an oscillation without long boot or ramp-up time but they sure don't improve the situation. Lazy loading of modules is a common cause, but any other resources that are loaded on demand or in the background. But that's more of a situation of thinking you need two additional nodes, getting three, settling back to two after warmup, and then killing them all off again when the spike ends. I may have said elsewhere that I'm more comfortable with scaling load based on daily and weekly cycle patterns, with perhaps a few exceptions for special events (sales, new initiatives, article publications) and making it easy for someone to dial up or down. To use a car analogy, get really good at power steering and power brakes before you attempt cruise control, get really good at cruise control before you attempt adaptive cruise control. And don't even talk about self-driving unless you're FAANG.
sethammons 7y ago
The load spikes here are predictable. They could spin up right before the extra resources are needed. However, it only masks the problem. Queue the argument for horizontal vs vertical scaling.

by btilly 7y ago

auslander 7y ago

> For example .. In fact this problem is so common .. So whats the problem here? Assuming there are no misconfigurations anymore, and just spiking loads.

btilly 7y ago
The problem is that people try to use autoscaling to handle a spiking load, and it fails abysmally. In my opinion for those with a spiking load profile, autoscaling should be considered harmful.

hinkley 7y ago

And god help you if that starts your autoscaler to flapping. Too much CPU load, start some servers. Too little, kill some servers. Oops now CPU is too high, bring them back. No, it's too low... It reminds me of the old UI bug where you get an off-by-one error for scrollbar logic and so it appears and disappears in a cycle until you resize. Edit: I think we want something akin to a learning thermostat, but for servers. Figure out that I get a CPU spike at 5:20 on week days and spin some servers up at 5:10. Then spin em down at 8:00 when EST starts to go to bed.

auslander 7y ago
> .. Oops now CPU is too high, bring them back. No, it's too low... You just described closed-loop control system oscillation. Cause is wrong gain and/or delay parameters. AWS ASG has some knobs for tuning like Cooldown, MaxSize. What you described above is most probably long boot time (delay) problem. Service should be ready in less than 30 secs after boot. To get there, one should bake AMIs, not install afresh all the stuff on boot.
- hinkley 7y ago
  You can get an oscillation without long boot or ramp-up time but they sure don't improve the situation. Lazy loading of modules is a common cause, but any other resources that are loaded on demand or in the background. But that's more of a situation of thinking you need two additional nodes, getting three, settling back to two after warmup, and then killing them all off again when the spike ends. I may have said elsewhere that I'm more comfortable with scaling load based on daily and weekly cycle patterns, with perhaps a few exceptions for special events (sales, new initiatives, article publications) and making it easy for someone to dial up or down. To use a car analogy, get really good at power steering and power brakes before you attempt cruise control, get really good at cruise control before you attempt adaptive cruise control. And don't even talk about self-driving unless you're FAANG.

sethammons 7y ago

The load spikes here are predictable. They could spin up right before the extra resources are needed. However, it only masks the problem. Queue the argument for horizontal vs vertical scaling.