3 ms·

> Can I Run DeepSeek R1 > Yes, you can run this model! Your system has sufficient resources (16GB RAM, 12GB VRAM) to run the smaller distilled version (likely

> Can I Run DeepSeek R1 > Yes, you can run this model! Your system has sufficient resources (16GB RAM, 12GB VRAM) to run the smaller distilled version (likely 7B parameters or less) of this model. Last I checked DeepSeek R1 was a 671B model, not a 7B model. Was this site made with AI?

sudohackthenews 2y ago
> > smaller distilled version Not technically the full R1 model, it’s talking about the distillations where Deepseek trained Qwen and Llama models based on R1 output
- do_not_redeem 2y ago
  Then how about DeepSeek R1 GGUF: > Yes, you can run this model! Your system has sufficient resources (16GB RAM, 12GB VRAM) to run this model. No mention of distillations. This was definitely either made by AI, or someone picking numbers for the models totally at random.
  - sudohackthenews 2y ago
    Ok yeah that’s just weird
reaperman 2y ago
It says “smaller distilled model” in your own quote which, generously, also implies quantized. Here[0] are some 1.5B and 8B distilled+quantized derivatives of DeepSeek. However, I don’t find a 7B model, that seems totally made-up from whole cloth. Also, I personally wouldn’t call this 8B model “DeepSeek”. 0: https://www.reddit.com/r/LocalLLaMA/comments/1iskrsp/quantized_deepseek_r1_distill_model_with_original/ https://www.reddit.com/r/LocalLLaMA/comments/1iskrsp/quantiz...
jsheard 2y ago
> Was this site made with AI? OP said they "vibe coded" it, so yes. https://en.m.wikipedia.org/wiki/Vibe_coding https://en.m.wikipedia.org/wiki/Vibe_coding
- kennysoona 2y ago
  Goodness. I love getting older and see the ridiculousness of the next generation.
monocasa 2y ago
Is it maybe because DeepSeek is a MoE and doesn't require all parameters for a given token? That's not ideal from a token throughput perspective, but I can see min working set of weight memory gains if you can load pieces into vram for each token.
- throwaway314155 2y ago
  It still wouldn't fit in 16 GB memory. Further there's too much swapping going on with MoE models to move expert layers to and from gpu without bottlenecks.
wbakst 2y ago
lol words out of my mouth

sudohackthenews 2y ago

> > smaller distilled version Not technically the full R1 model, it’s talking about the distillations where Deepseek trained Qwen and Llama models based on R1 output

do_not_redeem 2y ago
Then how about DeepSeek R1 GGUF: > Yes, you can run this model! Your system has sufficient resources (16GB RAM, 12GB VRAM) to run this model. No mention of distillations. This was definitely either made by AI, or someone picking numbers for the models totally at random.
- sudohackthenews 2y ago
  Ok yeah that’s just weird

reaperman 2y ago

It says “smaller distilled model” in your own quote which, generously, also implies quantized. Here[0] are some 1.5B and 8B distilled+quantized derivatives of DeepSeek. However, I don’t find a 7B model, that seems totally made-up from whole cloth. Also, I personally wouldn’t call this 8B model “DeepSeek”. 0: https://www.reddit.com/r/LocalLLaMA/comments/1iskrsp/quantized_deepseek_r1_distill_model_with_original/ https://www.reddit.com/r/LocalLLaMA/comments/1iskrsp/quantiz...

jsheard 2y ago

> Was this site made with AI? OP said they "vibe coded" it, so yes. https://en.m.wikipedia.org/wiki/Vibe_coding https://en.m.wikipedia.org/wiki/Vibe_coding

kennysoona 2y ago
Goodness. I love getting older and see the ridiculousness of the next generation.

monocasa 2y ago

Is it maybe because DeepSeek is a MoE and doesn't require all parameters for a given token? That's not ideal from a token throughput perspective, but I can see min working set of weight memory gains if you can load pieces into vram for each token.

throwaway314155 2y ago
It still wouldn't fit in 16 GB memory. Further there's too much swapping going on with MoE models to move expert layers to and from gpu without bottlenecks.

wbakst 2y ago

lol words out of my mouth