3 ms·
Disclaimer: I work at Exafunction I empathize a bit with the cloud providers as they have to upgrade their data centers every few years with new GPU instances
by pqn 4y ago
Disclaimer: I work at Exafunction
I empathize a bit with the cloud providers as they have to upgrade their data centers every few years with new GPU instances and it's hard for them to anticipate demand.
But if you can easily use every trick in the book (CPU version of the model, autoscaling to zero, model compilation, keeping inference in your own VPC, using spot instances, etc.) then it's usually still worth it.
- lowdose 4y agoNot to mention AWS has had a GPU cloud offering monopoly because Google Cloud and Microsoft Azure were publicly available until 2019.
- fomine3 4y agoGCP still provides NVIDIA K80. I wonder is it still worth to hold.
- varunkmohan 4y agoI think you'd probably always want to go with T4's since they are the same price unless there's just no availability for them.