3 ms·
2x CPU system may be slower for LLM than 1x CPU system. Because in 2x CPU system, the model may have to be passed via NUMA, which has 10% - 30% of memory bandw
by SlavikCA 2y ago
2x CPU system may be slower for LLM than 1x CPU system.
Because in 2x CPU system, the model may have to be passed via NUMA, which has 10% - 30% of memory bandwidth bandwidth
- ryao 2y agoIt would be interesting to see the performance if someone built a single socket version. I have a parts list here if anyone wants to try it: https://news.ycombinator.com/item?id=42868360 https://news.ycombinator.com/item?id=42868360