4 ms·
Wow, that's a huge feature. Thank you, guys. By the way, does anyone have a preferred case where they can put 4 AMD 7900XTX? There's a lot of motherboards and C
by renewiltord 3y ago
Wow, that's a huge feature. Thank you, guys. By the way, does anyone have a preferred case where they can put 4 AMD 7900XTX? There's a lot of motherboards and CPUs that support 128 lanes. It's the physical arrangement that I have trouble with.
- nottorp 3y agoUsed crypto mining parts not available any more?
- duskwuff 3y agoCrypto mining didn't require significant bandwidth to the card. Mining-oriented motherboards typically only provisioned a single lane of PCIe to each card, and often used anemic host CPUs (like Celeron embedded parts).
- renewiltord 3y agoExactly. They'd use PCIe x1 to PCIe x16 risers with power adapters. These require high-bandwidth.
- nottorp 3y agoOh. Shows I wasn't into that. I did once work with a crypto case, but yes, it was one motherboard with a lot of wifis and we still didn't need the pcie lanes.
- cjbprime 3y agoDoes LLM inference require significant bandwidth to the card? You have to get the model into VRAM, but that's a fixed startup cost, not a per-output-token cost.
- segmondy 3y agoYou don't need 128 lanes. 8x PCIe3 is more than enough, so for 4 cards that's 32. Most CPUs have about 40lanes. If you are not doing much that would be more than sufficient. Buy a PCIe riser. Go to amazon and search for it, a 16x to 16x PCIe riser. They go for about $25-$30 often about 20-30cm. If you want really long one, you can get one from China a 60cm for about the same price, you just have to wait for 3 weeks. That's what I did. Stuffing all those in a case is often difficult, so you have to go open rig. Either have the cables running out your computer and figuring out a way to support the cards while keeping them cool or just buy a small $20-$30 open rig frame.
- renewiltord 3y agoAh ha, that's the part I was curious about. I was wondering if I could keep everything cool open rig. I'm waiting for the stuff to arrive: risers, board, CPU, GPUs. And I've been putting it off because I wasn't sure how about the case. All right then, open rig frame. Thank you!
- Dylan16807 3y agoMost CPUs have about 20 lanes (plus a link to the chipset). On the one hand, they will be gen 4 or 5, so they're the equivalent of 40-80 gen 3 lanes. On the other hand, you can only split them up if you have a motherboard that supports bifurcation. If you buy the wrong model, you're stuck dedicating the equivalent of 64 gen 3 lanes to a single card. Edit: Actually, looking into it further, current Intel desktop processors will only run their lanes as 16(+4) or 8+8(+4). You can kind of make 4 cards work by using chipset-fed slots, but that sucks. You could also get a PCIe switch but those are very expensive. AMD will do 4+4+4+4(+4) on the right boards.
- segmondy 3y agoI suppose you're right, I don't know much about desktop cpus. I'm using cheap $10-$20 xeon E5-26XX CPUs which offer 40 lanes.