4 ms·
Yes, as GP said you'll need enough memory to load the weights (and KV cache). Beyond that it's just how fast you want to go. Pure CPU will be slow, but will wo
by magicalhippo 1y ago
Yes, as GP said you'll need enough memory to load the weights (and KV cache).
Beyond that it's just how fast you want to go. Pure CPU will be slow, but will work. There has been a few stories and comments here[1] about people running this model.
[1]: https://hn.algolia.com/?q=671B+DeepSeek https://hn.algolia.com/?q=671B+DeepSeek