4 ms·
The problem is that the specs of AMD consumer-grade GPUs do not translate to computer performance when you try and chain more than one together. I have 7 NVidi
by llm_trw 2y ago
The problem is that the specs of AMD consumer-grade GPUs do not translate to computer performance when you try and chain more than one together.
I have 7 NVidia 4090s under my desk happily chugging along on week long training runs. I once managed to get a Radeon VII to run for six hours without shitting itself.
- tspng 2y agoWow, are these 7 RTX 4090s in a single setup? Care to share more how you build it (case, cooling, power, ..)?
- adakbar 2y agoI'd like to know too
- ghxst 2y agoYou might find the journey of Tinycorp's Tinybox interesting, it's a machine with 6 to 8 4090 GPUs and you should be able to track down a lot of their hardware choices including pictures on their Twitter and other info on George his livestreams.
- llm_trw 2y agoBasically this but with an extra card on the x8 slot for connecting my monitors: https://www.youtube.com/watch?v=C548PLVwjHA https://www.youtube.com/watch?v=C548PLVwjHA There's a bunch of similar setups and there are a couple of dozen people that have done something similar on /r/localllama.
- osmarks 2y agoMost of these are just an EPYC server platform, some cursed risers and multiple PSUs (though cryptominer server PSU adapters are probably better). See https://nonint.com/2022/05/30/my-deep-learning-rig/ https://nonint.com/2022/05/30/my-deep-learning-rig/ and https://www.mov-axbx.com/wopr/wopr_concept.html https://www.mov-axbx.com/wopr/wopr_concept.html.
- icelancer 2y agoEPYC + Supermicro + C-Payne retimers/cabling. 208-240V power typically mandatory for the most affordable power supplies (chain a server/crypto PSU for the GPUs from ParallelMiner to an ATX PSU for general use). Beyond that, not much else.
- mpreda 2y ago> I have 7 NVidia 4090s under my desk I have 6 Radeon Pro VII under my desk (in a single system BTW), and they run hard for weeks until I choose to reboot e.g. for Linux kernel updates. I bought them "new old stock" for $300 apiece. So that's $1800 for all six.
- highwaylights 2y agoHow does the compute performance compare to 4090’s for these workloads? (I release it will be significantly lower, just try to get as much of a comparison as is possible).
- cainxinth 2y agoThe 4090 offers 82.58 teraflops of single-precision performance compared to the Radeon Pro VII's 13.06 teraflops.
- adrian_b 2y agoOn the other hand, for double precision a Radeon Pro VII is many times faster than a RTX 4090 (due to 1:2 vs. 1:64 FP64:FP32 ratio). Moreover, for workloads limited by the memory bandwidth, a Radeon Pro VII and a RTX 4090 will have about the same speed, regardless what kind of computations are performed. It is said that speed limitation by memory bandwidth happens frequently for ML/AI inferencing.
- ThinkBeat 2y agoWhat motherboard are you using to have space and ports for 7 of them?
- slavik81 2y agoThe ASRock Rack ROMED8-2T has seven PCIe x16 slots. They're too close together to directly put seven 4090s on the board, but you'd just need some riser cables to mount the cards on a frame.
- majke 2y agoWhat software stack you use for training?
- archi42 2y agoHow do you manage heat? I'm looking at a hashcat build with a few 5090, and water cooling seems to be the sensible solution if we scale beyond two cards.
- rcdwealth 2y ago[dead]