3 ms·
How many motherboards did you need to supply PCIe lanes to all of those P40s? I have a similarly-sized pile of MI25s and recently managed to get a eight of the
by crotchfire 3y ago
How many motherboards did you need to supply PCIe lanes to all of those P40s?
I have a similarly-sized pile of MI25s and recently managed to get a eight of them running in a single plentiful-on-ebay supermicro motherboard (custom bifurcating fanout-riser, x8 to each card). It was a "learning experience".
Falcon 180B, 4-bit quantization fits in VRAM and works, but
at under 10tokens/sec because llama.cpp's support for multi-GPU on AMD is kind of an afterthought.
- nullc 3y agoI have them across 4 hosts with pci-e switches. Really only 9 per host are maximally useful due to nvidia mailbox limits in cuda (also because one per host is on 8x, IIRC).
- crotchfire 3y ago> I have them across 4 hosts with pci-e switches. Nice! Sorry to pester you, last question: did you find a source for those at a reasonable price? Pricing from last time I investigated that would've been more than the P40s, so I'm assuming you did. Apparently the actual ICs are/were incredibly expensive and only Pericom sells them as standalone chips AFAICT. (Edit: oh wow, Pericom got bought by Diodes Inc; this explains a lot) Thanks!