4 ms·
And that's why you run models locally. Or if you want a remote chat model, use something stateless like AWS Bedrock custom model import to avoid having stored c
by NathanKP 2y ago
And that's why you run models locally. Or if you want a remote chat model, use something stateless like AWS Bedrock custom model import to avoid having stored chats on the server.
- dotancohen 2y agoNot many non-gamers have hardware capable of running such a model locally - never mind the skills. For most people, bash is not a tool for interacting with the computer, it is how they express their frustration with the computer (sometimes leaving damaged keyboards).
- loloquwowndueo 2y agoWow all the gamers with mad LLM skillz.
- 0x457 2y agoPretty sure gamers are mentioned because those are the usual demo that has GPUs with enough memory outside of people in the ML industry.
- loloquwowndueo 2y agoSo you’re in the demo scene as well? Yay
- 0x457 2y agoYou were not able to use context clues to figure that "demo" in this case is short for "demographics"? Sad
- razster 2y agoI have DeepSeek-R1 1.5b running on a Raspberry Pi 5. I have DS-R1 14b Q6 running on my old AM4 Ryzen with a AMD GPU, without issues. My primary workstation is running 32B Q8 and without issues. And it's simple!
- smallerize 2y agoThat's not the DeepSeek R1 model that they're offering via the API on these servers. That's a Qwen model that's been fine-tuned on output from the big R1 model.
- xinayder 2y agoSource?
- dreilide 2y agohttps://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-... DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Qwen-14B and DeepSeek-R1-Distill-Qwen-32B are derived from Qwen-2.5 series, which are originally licensed under Apache 2.0 License, and now finetuned with 800k samples curated with DeepSeek-R1. DeepSeek-R1-Distill-Llama-8B is derived from Llama3.1-8B-Base and is originally licensed under llama3.1 license. DeepSeek-R1-Distill-Llama-70B is derived from Llama3.3-70B-Instruct and is originally licensed under llama3.3 license.
- kgwgk 2y agohttps://arxiv.org/abs/2501.12948 https://arxiv.org/abs/2501.12948
- deleted 2y ago[deleted]
- tonygiorgio 2y agoYou could also use models that run on nvidia’s trusted execution environment.
- janalsncm 2y agoNvidia naming it “trusted” doesn’t mean I trust it.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]