2 ms·
I am reading from time to time about multi GPU solution and last time I found some real life information about this (it was two 7900 xtx) the result was that pe
by npodbielski 2y ago
I am reading from time to time about multi GPU solution and last time I found some real life information about this (it was two 7900 xtx) the result was that performance is the same at best often it is slower. So even if you manage to slap like 8 cheap cards onto motherboard, even if you would somehow make it work (people have problems with such setups), even if this would work continuously without much problems (crashes, power consumption) performance would be just OK.
I am not sure if spending 10k on such setup would be better than buying 10k card with 40gbs of RAM.
- pshirshov 2y agoOllama works fine with multi-gpu setups. Since rocm 6.3 everything is stable and you can mix different GPU generations. The performance is good enough for the models to be useful. The only thing which doesn't work well is running on iGPUs. It might work but it's very unstable.
- npodbielski 2y agoGood to know. Still, is it viable option? Buying i.e. AMD Threadripper for 2.5k$, motherboard and ram for 2k$ and i.e. 4 GPUs for 4k$ to have total of 96GB of VRAM? Total should be around 10k$ which is roughly price of Intel GPU specifically for AI if I am not mistaken? Which option would be better performance wise. I did never saw a comparison anywhere and this is too much money to make fun experiment over the weekend.
- zargon 2y ago> 10k$ which is roughly price of Intel GPU specifically for AI Huh? The largest vram card that Intel has is the A770 which is around $350. What exactly are you trying to compare against? Are you doing inference only or training?
- genewitch 2y agoIt can be read as a typo for "Nvidia" as their cards are about that price (or more, i haven't looked)