4 ms·
For people who don't want to click into X or don't have an account to see the thread: The rig in question uses dual-socket EPYCs and no GPUs, relying on 768 GB
by abound 2y ago
For people who don't want to click into X or don't have an account to see the thread: The rig in question uses dual-socket EPYCs and no GPUs, relying on 768 GB of fast DDR5 RAM. It gets about 6 - 8 tokens/sec for the full DeepSeek R1 model.
- lukan 2y agoThat Link is being discussed as well: https://news.ycombinator.com/item?id=42865575 https://news.ycombinator.com/item?id=42865575