3 ms·
How do the large clouds compare from an availability and cost perspective compared to finding a smaller provider and renting a dedicated cluster?
by ec109685 2y ago
How do the large clouds compare from an availability and cost perspective compared to finding a smaller provider and renting a dedicated cluster?
- choppaface 2y agoThe large clouds will often be able to give the biggest players a big discount. Perhaps not on raw GPU prices (maybe extended trial), but if you already have e.g. 1PB in object storage then they might give you 20-30% discount. Moreover if your contract is jumbo, they’ll give you not only a Slack channel but send Forward Deployed Engineers to your office and/or help you build part of the model training software. But for a deployment the size of the OP (Photoroom) I doubt any of the big clouds would offer a discount. Especially if they were not already negotiating with multiple clouds. Probably the best argument for going with a large cloud provider on a smaller budget is that you already use some of their other services significantly and your MLE-to-devops headcount makes something like Photoroom’s test infeasible.
- cavisne 2y agoLarge clouds tend to be not very good for GPU clusters. The security & management of multi-tenant GPU's is very complex. They have to buy from Nvidia so there is no negotiating leverage. And they know you will be a high maintenance customer (wanting all nodes to work at all times). So there is no big price or availability advantage for a large cloud (unless you are large enough to rent a dedicated cluster from a large cloud)