3 ms·
Do you actually need autoscaling? Or is it something you're worried you might need one day
by frankfrank13 4y ago
Do you actually need autoscaling? Or is it something you're worried you might need one day
- mattnewton 4y agoYes. 100% need to scale up the number of gpu workers and scale them back down based on request queue size, which is bursty. Otherwise we could spend 5 figures/month on gpus doing nothing for half the day and then still have unacceptable waits during traffic spikes