3 ms·

Noob Q: If I have 25000$ can I buy some GPUs and support the effort for supporting GPU Poor? Like is there a DIYer's guide to do this? I don't want to donate t

Noob Q: If I have 25000$ can I buy some GPUs and support the effort for supporting GPU Poor? Like is there a DIYer's guide to do this? I don't want to donate the cash but instead set up a handful of GPUs(Somehow?) and let people pay at cost to use them.

itissid 3y ago
Assume location is East Coast(NYC).
capableweb 3y ago
vast.ai can help you offer this and has bunch of docs on how you can start hosting: https://vast.ai/docs/hosting/hosting https://vast.ai/docs/hosting/hosting You won't be able to limit it to a particular set of clients who want to use it, but you're basically supporting the "GPU Poor" by doing this, as I think it's mostly smaller companies, individuals and researchers using vast.ai, rather than huge companies.
redox99 3y ago
25K will either get you money for renting servers, or allow you to jerry rig yourself something like a 8x4090 machine. The servers that most people would like to run (8xA100, 8xH100) are over 200k, and even if you had the money they're probably very hard to get right now.
- naillo 3y ago
  8x4090 machines should be a lot cheaper than that. 1599*8=12,792. I.e. closer to 16x4090 machines. Sure labor is worth something but he has it in cash, and also there's barely a lot of people (yet) paying high prices for research (non ML ops) labor hours for this yet so the salary wouldn't be that high anyway. Just a pet peeve when ppl under exaggerate how many gpus/flops a dollar can buy (if you're slighly smart with your money).
  - junipertea 3y ago
    What about PSU to power all of this, GPU motherboards, SSDs, CPU to not bottleneck this and sufficient RAM? You could argue it's more than 8, but definitely not 16. To utilize those GPUs to the max you need memory bandwidth to even feed it enough data.
    - fyloraspit 3y ago
      Yeh I built a dual 3090 machine and just to have those 2 cards run well cost a lot for all the other stuff. Workstation spec mobo, top tier PSU, CPU and memory with headroom for the system, not to mention storage and case and cooling. And this is just a hobbyist build. Cost would have easily doubled if I actually got server grade CPU and memory etc.
  - redox99 3y ago
    Meh, I was just ballparking. But add a threadripper, compatible motherboard, lots of RAM, fast storage, a few beast PSUs, and tax, and you're probably around 18k. So you ain't getting 2 of these machines for 25k. Plus you'll need to set aside some money for the power bill lol
    - bcatanzaro 3y ago
      Also possibly upgrading the wiring in your house or office.
rajko_rad 3y ago
honestly would recommend using spot/rented instances on that budget vs. buying! A100s run around ~$2/hour, the hard part is getting a large allocation... training workloads are quite spikey anyways, usually running lots of smaller experiments before actually doing a larger single run...
brailsafe 3y ago
No intention to attribute malice, but wouldn't this be more like buying a condo and renting it out to pay your mortgage rather than donating resources?
- rajko_rad 3y ago
  we didn't buy a condo haha, are letting people choose their own housing :)

itissid 3y ago

Assume location is East Coast(NYC).

capableweb 3y ago

vast.ai can help you offer this and has bunch of docs on how you can start hosting: https://vast.ai/docs/hosting/hosting https://vast.ai/docs/hosting/hosting You won't be able to limit it to a particular set of clients who want to use it, but you're basically supporting the "GPU Poor" by doing this, as I think it's mostly smaller companies, individuals and researchers using vast.ai, rather than huge companies.

redox99 3y ago

25K will either get you money for renting servers, or allow you to jerry rig yourself something like a 8x4090 machine. The servers that most people would like to run (8xA100, 8xH100) are over 200k, and even if you had the money they're probably very hard to get right now.

naillo 3y ago
8x4090 machines should be a lot cheaper than that. 1599*8=12,792. I.e. closer to 16x4090 machines. Sure labor is worth something but he has it in cash, and also there's barely a lot of people (yet) paying high prices for research (non ML ops) labor hours for this yet so the salary wouldn't be that high anyway. Just a pet peeve when ppl under exaggerate how many gpus/flops a dollar can buy (if you're slighly smart with your money).
- junipertea 3y ago
  What about PSU to power all of this, GPU motherboards, SSDs, CPU to not bottleneck this and sufficient RAM? You could argue it's more than 8, but definitely not 16. To utilize those GPUs to the max you need memory bandwidth to even feed it enough data.
  - fyloraspit 3y ago
    Yeh I built a dual 3090 machine and just to have those 2 cards run well cost a lot for all the other stuff. Workstation spec mobo, top tier PSU, CPU and memory with headroom for the system, not to mention storage and case and cooling. And this is just a hobbyist build. Cost would have easily doubled if I actually got server grade CPU and memory etc.
- redox99 3y ago
  Meh, I was just ballparking. But add a threadripper, compatible motherboard, lots of RAM, fast storage, a few beast PSUs, and tax, and you're probably around 18k. So you ain't getting 2 of these machines for 25k. Plus you'll need to set aside some money for the power bill lol
  - bcatanzaro 3y ago
    Also possibly upgrading the wiring in your house or office.

rajko_rad 3y ago

honestly would recommend using spot/rented instances on that budget vs. buying! A100s run around ~$2/hour, the hard part is getting a large allocation... training workloads are quite spikey anyways, usually running lots of smaller experiments before actually doing a larger single run...

brailsafe 3y ago

No intention to attribute malice, but wouldn't this be more like buying a condo and renting it out to pay your mortgage rather than donating resources?

rajko_rad 3y ago
we didn't buy a condo haha, are letting people choose their own housing :)