4 ms·
Why not buy 2x24GB cards and use SLI with CUDA? Or does that represent a bottleneck-so-bad that it's a dealkiller and worth the $3000 premium?
by DaiPlusPlus 3y ago
Why not buy 2x24GB cards and use SLI with CUDA? Or does that represent a bottleneck-so-bad that it's a dealkiller and worth the $3000 premium?
- wmf 3y agoSLI no longer exists.
- imtringued 3y agoLlama.cpp wasn't optimized for multi GPU setups. You need to split individual layers onto the GPU instead of simple offloading.