2 ms·

So should we buy CoreWeave and Lambda shares after IPO or not? For enthusiasts, even renting from Runpod, Salad, Vast.ai which are an order of magnitude cheape

So should we buy CoreWeave and Lambda shares after IPO or not? For enthusiasts, even renting from Runpod, Salad, Vast.ai which are an order of magnitude cheaper than established cloud providers is much more expensive than buying a Rtx 3090 or 4090. Which got me thinking why don't companies who need training don't put their money together, buy some H100 and share them?

Hewlberno 3y ago
They will when federated learning kicks off
mschuster91 3y ago
> Which got me thinking why don't companies who need training don't put their money together, buy some H100 and share them? Because that's a lot of capex to take on, not to mention accounting hassles, depreciation, and install/maintenance expenses. Your average bunch of startups does not want to tie up a giant chunk of money in hardware, and even most established companies don't want to either, hence the giant "move to the cloud" epidemic of the last years. On top of that, you need some serious expertise to properly wire up an AI training cluster so you can actually use the GPUs to their full potential, and such experts are rare. The problem is that MBA beancounters don't have a way to put a number behind "what if the cloud provider suddenly decides to terminate us", "what if creds leak and someone mines shitcoins for a week or two", or (for everyone outside FVEY) "what if the NSA decides to compel the cloud provider to turn over our server data containing our trade secrets", and so this gets all ignored.
- latchkey 3y ago
  This is a good answer. Don't forget that H100 class compute is US export controlled and there are some hard limitations we sign agreements on, talking about what they can and cannot be used for (like building bombs and such). CSP's are taking on that KYC risk too.
Der_Einzige 3y ago
Fat citation needed on "order of magnitude cheaper". OCI is a hyperscaler/established provider and offers GPUs for quite cheap compared to the others. They are competitive with Runpod, Vast, and Salad, especially if you negotiate (which everyone who knows this business does).

Hewlberno 3y ago

They will when federated learning kicks off

mschuster91 3y ago

> Which got me thinking why don't companies who need training don't put their money together, buy some H100 and share them? Because that's a lot of capex to take on, not to mention accounting hassles, depreciation, and install/maintenance expenses. Your average bunch of startups does not want to tie up a giant chunk of money in hardware, and even most established companies don't want to either, hence the giant "move to the cloud" epidemic of the last years. On top of that, you need some serious expertise to properly wire up an AI training cluster so you can actually use the GPUs to their full potential, and such experts are rare. The problem is that MBA beancounters don't have a way to put a number behind "what if the cloud provider suddenly decides to terminate us", "what if creds leak and someone mines shitcoins for a week or two", or (for everyone outside FVEY) "what if the NSA decides to compel the cloud provider to turn over our server data containing our trade secrets", and so this gets all ignored.

latchkey 3y ago
This is a good answer. Don't forget that H100 class compute is US export controlled and there are some hard limitations we sign agreements on, talking about what they can and cannot be used for (like building bombs and such). CSP's are taking on that KYC risk too.

Der_Einzige 3y ago

Fat citation needed on "order of magnitude cheaper". OCI is a hyperscaler/established provider and offers GPUs for quite cheap compared to the others. They are competitive with Runpod, Vast, and Salad, especially if you negotiate (which everyone who knows this business does).