3 ms·
$1,961.19: H13SSL-N Motherboard And EPYC 9334 QS CPU + DDR5 4*128GB 2666MHZ REG ECC RAM Server motherboard kit https://www.aliexpress.us/item/3256807766813460.
by timschmidt 2y ago
$1,961.19: H13SSL-N Motherboard And EPYC 9334 QS CPU + DDR5 4*128GB 2666MHZ REG ECC RAM Server motherboard kit
https://www.aliexpress.us/item/3256807766813460.html https://www.aliexpress.us/item/3256807766813460.html
Doesn't seem like 20x to me. I'm sure spending more than 30 seconds searching could find even better deals.
- pixelpoet 2y agoIsn't 2666 MHz ECC RAM obscenely slow? 32 cores without the fast AVX-512 of Zen5 isn't what anyone is looking for in terms of floating point throughput (ask me about electricity prices in Germany), and for that money I'd rather just take a 4090 with 24GB memory and do my own software fixed point or floating point (which is exactly what I do personally and professionally). This is exactly what I meant about Intel's recent launch. Imagine if they went full ALU-heavy on latest TSMC process and packaged 128GB with it, for like, 2-3k Eur. Nvidia would be whipping their lawyers to try to do something about that, not just their engineers.
- timschmidt 2y agoI don't think anyone's stopping you, buddy. Great chat. I hope you have a nice evening.
- ryao 2y agoYes and no. I have been developing some local llama 3 inference software on a machine with 3200MT/s ECC RAM and a Ryzen 7 5800X: https://github.com/ryao/llama3.c https://github.com/ryao/llama3.c My experience is that input processing (prompt processing) is compute bottlenecked in GEMM. AVX-512 would help there, although my CPU’s Zen 3 cores do not support it and the memory bandwidth does not matter very much. For output generation (token generation), memory bandwidth is a bottleneck and AVX-512 would not help at all.