6 ms·
Ah, we're running a medium amount of compute at zero-margin. The point is not to go sell the Fortune 500, but to make sure a grad student can spend a $50k grant
by flaque 3y ago
Ah, we're running a medium amount of compute at zero-margin. The point is not to go sell the Fortune 500, but to make sure a grad student can spend a $50k grant.
Right now, it's pretty easy to get a few A/H100s (Lambda is great for this), but very hard to get more than 24 at a reasonable price ($~2 an hour). One often needs to put up a 6+ month commitment, even when they may only want to run their H100s for an 8 hour training run.
It's the right business decision for GPU brokers to do long term reservations and so on, and we might do so too if we were in their shoes. But we're not in their shoes and have a very different goal: arm the rebels! Let someone who isn't BigCorp train a model!
- trostaft 3y ago> but to make sure a grad student can spend a $50k grant. As a graduate student, thank you. Thankfully, my workloads aren't LLM crazy so I can get by on my old NVIDIA consumer hardware, but I have coworkers struggling to get reasonable prices/time for larger scale hardware.
- lulunananaluna 3y agoThis is great. Thank you very much for your work.
- narrator 3y agoSo what happens when some big bucks VC backed closed source LLM company buys all your compute inventory for the next 5 years? This is not that unlikely. Lambda Labs a little while back was completely sold out of all compute inventory.
- xeromal 3y agoI assume it's up to them to say no. They did say they're not in it to make bookoo bucks
- stavros 3y agoI must say, this is the worst I've seen "beaucoup" spelled.
- xeromal 3y agoHahahaha. I can honestly say no on has told me in my entire uneducated life that "bookoo" was a real word. I appreciate the lesson.
- agajews 3y agoYeah we aren’t going to let anyone book the whole thing for years. If we ever have to make the choice, we’ll choose the startups over the big companies.
- flaque 3y agoYeah, if someone doesn't care about the cost and wants to buy whole cluster, they might be better off using an existing provider.
- gnopgnip 3y agoPresumably them buy more gpus
- ec109685 3y agoHow can you allow people to get big chunks of GPU’s without a lot of expensive slack in the system?