While all the things you mentioned are correct, I would also like to point out one of the fundamental results from control theory here: The time period of the c

While all the things you mentioned are correct, I would also like to point out one of the fundamental results from control theory here: The time period of the control loop must be proportionate to the time it takes for any changes to the controlled system to propagate, otherwise you will get wild oscillations (in the number of instances in this case). Since it can take a few minutes for an EC2 instance to fully boot up, start up all the applications it needs and start serving traffic, you can't have a very fast autoscaler loop adding a new instance every second: it would start up way more instances than needed and eventually have to shut most of them down again, etc. This EC2 startup delay is the core problem, if you could reduce that interval the autoscaler could run way faster even with the usual distributed HA, CAP theorem and metrics concerns.