4 ms·

This applies for all neural networks. Depending on how much money you're willing to spend, in descending order: DGX (computer with 8 A100s, $150,000), A100 (80G

This applies for all neural networks. Depending on how much money you're willing to spend, in descending order: DGX (computer with 8 A100s, $150,000), A100 (80GB, $15,000), A6000 ($5000), RTX 3090 ($1000).

moyix 4y ago
I just grabbed an A6000 for $3500 on eBay, so you can probably get a pretty decent deal on those now. They're pricey but IMO it's a great deal if you really need the VRAM (e.g. for training LLMs).
hedora 4y ago
I'd consider renting these, especially if you are just getting started: https://docs.paperspace.com/core/compute/machine-types/ https://docs.paperspace.com/core/compute/machine-types/ 1 x A100, 80GB is $3.19 / hour. 8 x A100, 80GB is about $25 / hour. They have much less expensive machines. I used to use them to run steam games, but now proton is just too darn good. Their low end machines are OK for CAD software that supports real-time raytracing.
- asciimike 4y ago
  Lots of good options for cheap GPU clouds, including Paperspace (mentioned above), Coreweave, and Crusoe Cloud (crusoecloud.com). Crusoe Cloud's angle is that our GPUs are powered off otherwise wasted energy and are carbon-reducing; running one for a year provides an emissions reduction equivalent to taking one car off the road. Disclosure, I'm head of product @ Crusoe Cloud. Feel free to ping me at mike at crusoecloud dot com if you've got questions or feedback.
  - pbronez 4y ago
    Cool idea. Tried to join your waitlist but the form throws errors. Could be my security measures, but I’m on mobile safari, toggled off all content blockers and still had an error.
    - asciimike 4y ago
      Odd, I just tested on Safari mobile and it went through without issues. Mind sending me an email at mike at crusoecloud dot com and we can get you set up?
  - douto 4y ago
    How can this be carbon-reducing? That sounds like a dubious marketing claim.
    - asciimike 4y ago
      TL;DR: we run data centers on-site at oil wells and take natural gas that would otherwise be flared (it can't be economically transported as natural gas in a pipeline or turned into electricity and transported) and combust it completely. Methane is a significantly more potent greenhouse gas than CO2, so it ends up being a net reduction in emissions vs what's currently happening. https://www.crusoeenergy.com/digital-flare-mitigation https://www.crusoeenergy.com/digital-flare-mitigation has some more information.
      - ip26 4y ago
        Is flare combustion more incomplete than your generation?
      - brimwats 4y ago
        on the other hand, does that mean you're obligated to be in oil producing countries? how does the geopolitics of that work?
  - hedora 4y ago
    How do you carbon offset? We have a bunch of diseased pine trees. Assuming we don't have enough time to cut them down and burn them in the next 12 months, and that I am unscrupulous, I could sell you carbon credits through one of the bigger exchanges. You would never know, and it would be completely above board. John Oliver had a particularly depressing segment on this recently. (Totally off topic, but I'd love to biochar them instead. Using them for lumber and shipping them offsite are non-starters. Any ideas, anyone?)
    - asciimike 4y ago
      See above post, but the TL;DR: is "we capture methane that would otherwise be flared." Agreed that a lot of carbon offsets look like, "we were going to cut this section of the rainforest down, but if you pay us, we won't do that for X period of time." This is _actually_ reducing existing emissions.
- lbotos 4y ago
  As someone who just fired off both paperspace boxes and coreweave to play around with stable diffusion, Paperspace limited me to some old janky hardware and never replied to my request to "unlock" the more modern cards. Coreweave let me get a modern card box first go round. I'd go with coreweave if you are a weekend hacker who just wants access to a lil bit of beefy GPU. YMMV.
- cwkoss 4y ago
  Are there "mining pools" for running AI jobs and getting paid in crypto?
  - knicholes 4y ago
    There's vast.ai, but you get paid in USD.
UncleOxidant 4y ago
Won't 3090 prices go down a good bit after the 4xxx series cards start coming out in a couple of months?
- kajecounterhack 4y ago
  My guess is "not as much as most folks seem to think" Comparable 24gb vram 4xxx cards are also 1 slot bigger and many watts hungrier. If you want to be able to use a 800w PSU + 3 slots, and just need 24gb vram for say, running inference on a big diffusion model, then 3090 is still going to be your only option for a while. They're pretty well priced right now, at $1k. If you need one, not much reason to wait, your time is probably worth more than saving a couple hundred bucks.
  - KaoruAoiShiho 4y ago
    Is there no consumer option bigger than 24gb next gen?
    - InvaderFizz 4y ago
      It appears as though the 4090 will be 24GB, but that card may also be almost $2k. Used 3090s on eBay are $800 all day long. That price may drop a bit in the next week or so, but not much, as that 24GB of VRAM is the main draw for that over a 3080Ti.
    - tootyskooty 4y ago
      The 4090ti is rumoured to be 48GB [1], but who knows when that will release or how much it will cost. If you really need extra VRAM and don't mind longer inference, older used Tesla cards are an option. A used Tesla V100 32GB can be sometimes found on Ebay for 1500. [1] https://www.techpowerup.com/gpu-specs/geforce-rtx-4090-ti.c3917 https://www.techpowerup.com/gpu-specs/geforce-rtx-4090-ti.c3...
      - KaoruAoiShiho 4y ago
        Thank you great tip on V100.

moyix 4y ago

I just grabbed an A6000 for $3500 on eBay, so you can probably get a pretty decent deal on those now. They're pricey but IMO it's a great deal if you really need the VRAM (e.g. for training LLMs).

hedora 4y ago

I'd consider renting these, especially if you are just getting started: https://docs.paperspace.com/core/compute/machine-types/ https://docs.paperspace.com/core/compute/machine-types/ 1 x A100, 80GB is $3.19 / hour. 8 x A100, 80GB is about $25 / hour. They have much less expensive machines. I used to use them to run steam games, but now proton is just too darn good. Their low end machines are OK for CAD software that supports real-time raytracing.

asciimike 4y ago
Lots of good options for cheap GPU clouds, including Paperspace (mentioned above), Coreweave, and Crusoe Cloud (crusoecloud.com). Crusoe Cloud's angle is that our GPUs are powered off otherwise wasted energy and are carbon-reducing; running one for a year provides an emissions reduction equivalent to taking one car off the road. Disclosure, I'm head of product @ Crusoe Cloud. Feel free to ping me at mike at crusoecloud dot com if you've got questions or feedback.
- pbronez 4y ago
  Cool idea. Tried to join your waitlist but the form throws errors. Could be my security measures, but I’m on mobile safari, toggled off all content blockers and still had an error.
  - asciimike 4y ago
    Odd, I just tested on Safari mobile and it went through without issues. Mind sending me an email at mike at crusoecloud dot com and we can get you set up?
- douto 4y ago
  How can this be carbon-reducing? That sounds like a dubious marketing claim.
  - asciimike 4y ago
    TL;DR: we run data centers on-site at oil wells and take natural gas that would otherwise be flared (it can't be economically transported as natural gas in a pipeline or turned into electricity and transported) and combust it completely. Methane is a significantly more potent greenhouse gas than CO2, so it ends up being a net reduction in emissions vs what's currently happening. https://www.crusoeenergy.com/digital-flare-mitigation https://www.crusoeenergy.com/digital-flare-mitigation has some more information.
    - ip26 4y ago
      Is flare combustion more incomplete than your generation?
    - brimwats 4y ago
      on the other hand, does that mean you're obligated to be in oil producing countries? how does the geopolitics of that work?
- hedora 4y ago
  How do you carbon offset? We have a bunch of diseased pine trees. Assuming we don't have enough time to cut them down and burn them in the next 12 months, and that I am unscrupulous, I could sell you carbon credits through one of the bigger exchanges. You would never know, and it would be completely above board. John Oliver had a particularly depressing segment on this recently. (Totally off topic, but I'd love to biochar them instead. Using them for lumber and shipping them offsite are non-starters. Any ideas, anyone?)
  - asciimike 4y ago
    See above post, but the TL;DR: is "we capture methane that would otherwise be flared." Agreed that a lot of carbon offsets look like, "we were going to cut this section of the rainforest down, but if you pay us, we won't do that for X period of time." This is _actually_ reducing existing emissions.
lbotos 4y ago
As someone who just fired off both paperspace boxes and coreweave to play around with stable diffusion, Paperspace limited me to some old janky hardware and never replied to my request to "unlock" the more modern cards. Coreweave let me get a modern card box first go round. I'd go with coreweave if you are a weekend hacker who just wants access to a lil bit of beefy GPU. YMMV.
cwkoss 4y ago
Are there "mining pools" for running AI jobs and getting paid in crypto?
- knicholes 4y ago
  There's vast.ai, but you get paid in USD.

UncleOxidant 4y ago

Won't 3090 prices go down a good bit after the 4xxx series cards start coming out in a couple of months?

kajecounterhack 4y ago
My guess is "not as much as most folks seem to think" Comparable 24gb vram 4xxx cards are also 1 slot bigger and many watts hungrier. If you want to be able to use a 800w PSU + 3 slots, and just need 24gb vram for say, running inference on a big diffusion model, then 3090 is still going to be your only option for a while. They're pretty well priced right now, at $1k. If you need one, not much reason to wait, your time is probably worth more than saving a couple hundred bucks.
- KaoruAoiShiho 4y ago
  Is there no consumer option bigger than 24gb next gen?
  - InvaderFizz 4y ago
    It appears as though the 4090 will be 24GB, but that card may also be almost $2k. Used 3090s on eBay are $800 all day long. That price may drop a bit in the next week or so, but not much, as that 24GB of VRAM is the main draw for that over a 3080Ti.
  - tootyskooty 4y ago
    The 4090ti is rumoured to be 48GB [1], but who knows when that will release or how much it will cost. If you really need extra VRAM and don't mind longer inference, older used Tesla cards are an option. A used Tesla V100 32GB can be sometimes found on Ebay for 1500. [1] https://www.techpowerup.com/gpu-specs/geforce-rtx-4090-ti.c3917 https://www.techpowerup.com/gpu-specs/geforce-rtx-4090-ti.c3...
    - KaoruAoiShiho 4y ago
      Thank you great tip on V100.