2 ms·
Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.
by codedokode 3mo ago
Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.
- nicce 3mo ago> the only problem you need 8 DRAM controllers. Which is a huge problem? Even using 2x memory controllers in typical consumer motherboard can make system very unstable.
- codedokode 3mo agoWhy is that so?
- deleted 3mo ago[deleted]
- jaxytee 3mo agoRunning large MOE models won't saturate the EPYC memory. You'll never see that 200 Gb/s running inference.
- edg5000 3mo agoBecause it'll bottleneck at the CPU? Or because 200 bB/s is overkill when running a model (that fits in memory)?