3 ms·
You are correct, except it's not that hard to run locally. Here, somebody made a 6k rig to do it: https://x.com/carrigmat/status/1884244369907278106 https://x.c
by deeviant 2y ago
You are correct, except it's not that hard to run locally. Here, somebody made a 6k rig to do it: https://x.com/carrigmat/status/1884244369907278106 https://x.com/carrigmat/status/1884244369907278106
- abound 2y agoFor people who don't want to click into X or don't have an account to see the thread: The rig in question uses dual-socket EPYCs and no GPUs, relying on 768 GB of fast DDR5 RAM. It gets about 6 - 8 tokens/sec for the full DeepSeek R1 model.
- lukan 2y agoThat Link is being discussed as well: https://news.ycombinator.com/item?id=42865575 https://news.ycombinator.com/item?id=42865575