3 ms·
Isn't DeepSeek simple/small enough you can run it locally?
by marcodiego 2y ago
Isn't DeepSeek simple/small enough you can run it locally?
- deepsquirrelnet 2y agoAt least a TB of VRAM to load it in fp16. They distilled to smaller models, which do not perform as well, but can be run on a single GPU. Full R1 is big though.
- nickthegreek 2y agofp16? I thought it was trained at fp8.
- deleted 2y ago[deleted]
- zbendefy 2y agoNo, the full R1 model is ~650GB. There are quantized version that quantize it down to ~150GB. What you can run locally are the distilled models, that is actually LLama and Qwen weights further trained on R1's output