3 ms·
Cold start time is the issue with most serverless runtimes. Your own mission statement states: "We want on-demand, granular elastic scale of specific parts of
by solatic 3y ago
Cold start time is the issue with most serverless runtimes.
Your own mission statement states: "We want on-demand, granular elastic scale of specific parts of our app code." Doing that correctly is fundamentally a question of how long you need to wait for cold starts, because if you have a traffic spike, the spiked part of the traffic is simply not being served until the cold start period elapses. If you're running hot runners with no load, or if you have incoming load without runners (immediately) serving them, then you're not really delivering on your goal here. AWS EC2 has had autoscaling groups for more than a decade, and of course, a VM is essentially a more elaborate wrapper for any kind of application code you can write, and one with a longer cold-start time.
> Under load you are rarely paying the cold start time because the pool is already hot.
My spiky workloads beg to differ.
- bo0tzz 3y agoDepending of course on the workload and request volume, I imagine you could apply a strategy where code is run locally while waiting for a remote node to start up, so you can still serve the requests on time?
- solatic 3y agoNo, because then you're dividing the resources allocated to the function among the existing run + the new run. If you over-allocate ahead of time to accommodate for this, you might as well just run ordinary VMs, which always have excess allocation locally; the core idea of scaling granularly is that you only allocate the resources you need for that single execution (paying a premium compared to a VM but less overall for spiky workloads since less overhead will be wasted).
- conradfr 3y agoIn a Elixir/Phoenix app I don't think this will be really used for web traffic and more for background/async jobs.