4 ms·
The bandwidth still doesn't quite compare to a GPU, and 128GB doesn't fit DeepSeek R1, however. If they bump it to 512GB for $5000 or so, that will disrupt the
by alwayslikethis 2y ago
The bandwidth still doesn't quite compare to a GPU, and 128GB doesn't fit DeepSeek R1, however. If they bump it to 512GB for $5000 or so, that will disrupt the market.
- ricardobeat 2y ago256GB/s is on par with the M4 Pro at a much lower price. You could run R1 671B using unsloth’s quantized version that fits in <80GB. Not sure why that would be a benchmark though, there’s nothing that can run the model at full precision right now except for (very slow) server hardware.
- deleted 2y ago[deleted]
- ant6n 2y agoBut you can use an AMD EPYC cpu and get 460GB/s bandwidth. Could probably get 512GB RAM for a lot cheaper than 5K.