3 ms·
what's the point of 512GB RAM for LLMs on this Mac Studio if the speed is painfully slow? You can fit the entire Deepseek 671B q4 into this computer and get 41
by aurareturn 2y ago
what's the point of 512GB RAM for LLMs on this Mac Studio if the speed is painfully slow?
You can fit the entire Deepseek 671B q4 into this computer and get 41 tokens/s because it's an MoE model.
- KingOfCoders 2y agoYour comments went from "40 tokens/s by my calculations" to "40 tokens/s" to "41 tokens/s" Is there a dice involved in "your calculations?"
- aurareturn 2y ago41 was when I learned it has a little over 800B/s. Doesn't matter. All theorized because no one has publicly tested one.