3 ms·
I ran the 32b parameter model just fine on my rig an hour ago with a 4090 and 64gig of ram. It’s high end for the consumer scene but still solidly within consum
by lovich 2y ago
I ran the 32b parameter model just fine on my rig an hour ago with a 4090 and 64gig of ram. It’s high end for the consumer scene but still solidly within consumer prices
- mv4 2y agoI have also been running the 32b version on my 24GB RTX 3090.
- rocho 2y agoThat's not DeepSeek, it's a Qwen or Llama model distilled from DeepSeek. Not the same thing at all.
- mv4 2y agoif someone wants to run the real thing (R1) locally, someone posted their hardware specs on X. Total cost: $6,000. [0] direct link with login https://x.com/carrigmat/status/1884244369907278106 https://x.com/carrigmat/status/1884244369907278106 [1] alt link without login https://threadreaderapp.com/thread/1884244369907278106.html https://threadreaderapp.com/thread/1884244369907278106.html
- testrun 2y agoI am doing the same.
- seanmcdirmid 2y agoI'm confused. According to other comment: https://news.ycombinator.com/item?id=42859645 https://news.ycombinator.com/item?id=42859645, <= 70b DeepSeek models are just a fine tuning of Llama or Qwen? So we shouldn't take any thought of these models to actually being DeepSeek. I think people are confusing the smaller non-DeepSeek original models (Qwen/Llama) with the 700B DeepSeek R1 model being talked about in here and that very few people can run locally.
- rurban 2y agoI run the 32b parameter model also just fine on our 4x H100 rig :) It's good enough for embedding, our use-case.
- plaguuuuuu 2y agoI'm not sure if $200k of hardware fits the consumer level