4 ms·
We specifically colo GPU hardware due to how expensive it is. For instance this year we spent ~$500k (rough estimate) on purchasing GPU hardware. With cloud th
by cardine 5y ago
We specifically colo GPU hardware due to how expensive it is.
For instance this year we spent ~$500k (rough estimate) on purchasing GPU hardware. With cloud that would have costed us ~$80k/mo - so after ~6 months we break even on the hardware and then after that just have to spend the much smaller colo bill. We expect hardware to last us 3+ years so we come out far ahead by buying.
Our work is so specialized anyways that if we went cloud we would still have to have the same dev ops staff anyways so there is not much cost difference on that side of things. We had to invest a little bit more upfront in deployment, but in the end it is peanuts compared to the $80k/mo that we are saving in cloud costs.
The only trouble is when we need to scale beyond our currently owned hardware. In that case, once we are above capacity of servers we own, we make another purchase and in the interim use cloud to bridge the gap.
In the end purchasing hardware ourselves is one of the highest ROI moves we have made and I would highly recommend it to others. Of course if your cloud bill is much lower you might find the cost savings to not be enough to be worth the hassle.
- 71a54xd 5y agoYep, this is exactly why I'm looking to colo. I have eight or so RTX a5000 gpu's and although colo is expensive, it's an absolute slam dunk in comparison to renting from AWS or even "dedicated" companies that only rent out GPU hardware at slightly better rates. I will say, power and efficiency become a bigger issue when colocating - if our scaling requirements and credit were slightly different we'd likely have preferred a greater up front cost and purchased V100 gpus instead of RTX a5000's.
- cardine 5y agoI'd recommend Colovore for power - they are specialized for this use case and do a great job for us. We started in the same situation where we had 10x RTX 8000 cards. We've since added tons of A6000 and A100 cards in our cabinet and have had no power issues.