3 ms·
GPUs in 16x slots is still important for LLM stuff, especially multi-GPU, where lots of data needs to move between cards during computation.
by hoss1474489 11mo ago
GPUs in 16x slots is still important for LLM stuff, especially multi-GPU, where lots of data needs to move between cards during computation.
- Dylan16807 11mo agoDepends on what you're doing. I'm pretty sure the bandwidth for inference isn't much.
- eurekin 11mo agoDepends, if it's tensor parallel or pipeline parallel. Only PP doesn't pass too much. TP does
- __alexs 11mo agoA 16x PCIE 6.0 setup has more bandwidth than any dual channel DDR5 memory kit.