> But whenever you have a burst of scale and Lambda needs to spin up more instances... some of your unlucky users are going to hit that cold start. And this hac

> But whenever you have a burst of scale and Lambda needs to spin up more instances... some of your unlucky users are going to hit that cold start. And this hack of keeping an instance warm would do nothing to solve that. I've been doing a bunch of reading as a side-effect of being in and around the riff and Knative autoscaler efforts. What you're describing is known in other professions as a "stock-out". The good news is: there are existing models for answering this kind of question. From what I've seen the "order up to" model is a fit, but I've yet to find time to work on testing that theory. The bad news is: this problem never goes away. You are always going to be oversupplied or undersupplied. Autoscalers don't let you break Little's Law or overturn causality. The good or bad news, depending on how you think: this tradeoff can be tuned. You can choose an acceptable probability of running out of running instances vs the acceptable average level of utilisation. That tradeoff is purely economic, it is a business decision, not an engineering decision. An autoscaler cannot throw the bones, gaze into the crystal ball and mystically divine your intentions. A human will still be responsible for the decisions that matter.