5 ms·
All you need is 4x 4090 GPUs to Train Your Own Model -- and $12000 to buy them
by master_crab 2y ago
All you need is 4x 4090 GPUs to Train Your Own Model -- and $12000 to buy them
- sabareesh 2y agoI was contemplating between building rig vs using the cloud but for some reason I want to get hands on. So you can always rent them for a fraction of cost
- mcphage 2y agoWell, one or the other, at any rate :-)
- bfung 2y agoAlso (at least in Southern California) electricity prices and how long the rig is on. Not as bad as the initial build cost, but run costs will add up over time.
- echelon 2y agoYou can get a used A100 for that cost and have better software support for training. 4090s are too small for training and you'll have to write your own suboptimal batching. Unless you value the learning, it'd be better to rent GPUs in the cloud for training.
- kristopolous 2y agoThe GPU rental market is fairly reasonable. There's lots of companies doing it. (I work at one of them). 4x 4090 can be fetched for around $0.40/hour on some platforms ... about $1.20 on others depending on how available you want it. Regardless, all in, you can do an average 10-or-so-day train for < $500. If you want on-prem, wait a few months. The supply of 5000 series (probably announced at CES in a few days) should push more 4000 on the market and, maybe, for a bit, over-supply and push the price down. Nvidia stopped manufacturing the 4000 a few months ago because they don't have endless factories. Those resources were reallocated to 5000 series and thus pushed the price for the 4000 up to the ridiculous place it is now (about $2,000 on ebay) I think the current appetite for crypto and ai is big enough to consume all 4000 and 5000 series cards to a point of scarcity (even 3090s are still fetching about $1000) but there should be a window where things aren't crazy expensive coming up. There's no evidence supply will continually outstrip demand unless something unusual happens.
- whimsicalism 2y agodon't you need nvlink? feel like an 80gb a100 would start being worth it at a $1.20/4x 4090 price point
- kristopolous 2y agoSome suppliers have support for it, some don't. They either use docker or kvm and it depends on how clever their hosting software is. We can do it, but that's a recent thing. it's really hit or miss
- whimsicalism 2y ago? sorry i really don't understand this reply... some suppliers have support for nvlink on 4090? i doubt that
- deleted 2y ago[deleted]
- yieldcrv 2y ago
- andrewmcwatters 2y agoThe last time I checked, a modern Threadripper build is a bit over $10,000. So if you have the budget for that but need something GPU-oriented instead, then I could see that being a reasonable option.
- KeplerBoy 2y agoThe thing is you need a threadripper-class build to make use of 4 GPUs in the first place. Ordinary PCs don't have the PCIe lanes necessary for that. But pricing is okay-ish, have a look at Geohot's Tinybox for turnkey solutions.
- andrewmcwatters 2y agoAh, of course. I forgot about PCI-e lane requirements. Yeah, you're not going to casually find 8-slot (12?) PCIe x16 motherboard configurations.
- Dylan16807 2y agoHow much PCIe bandwidth do you need to avoid it being the bottleneck?
- KeplerBoy 2y agoDepends on the Application. In Bitcoin farming it famously was not an issue at all, manufacturers came up with the weirdest motherboards featuring many x1 pcie slots. Look up the Biostar TB360-BTC PRO 2.0 if you want to see a curiosity. In Deep Learning it depends on your sharding strategy.
- yieldcrv 2y agoThats way less than the 6 or 7 figure sums from a year ago I’m glad to know