12 ms·
Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.
by codedokode 5mo ago
Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.
- nicce 5mo ago> the only problem you need 8 DRAM controllers. Which is a huge problem? Even using 2x memory controllers in typical consumer motherboard can make system very unstable.
- codedokode 5mo agoWhy is that so?
- deleted 5mo ago[deleted]
- jaxytee 5mo agoRunning large MOE models won't saturate the EPYC memory. You'll never see that 200 Gb/s running inference.
- edg5000 5mo agoBecause it'll bottleneck at the CPU? Or because 200 bB/s is overkill when running a model (that fits in memory)?