4 ms·
You’ll be able to run distills of DeepSeek R1. For the full model, you can network together multiple and use exo or llama.cpp RPC. You can do image generation
by nrp 2y ago
You’ll be able to run distills of DeepSeek R1. For the full model, you can network together multiple and use exo or llama.cpp RPC.
You can do image generation as well! AMD has some demos of this, and they are maturing PyTorch support to broaden the software ecosystem.
- baobabKoodaa 2y agoThanks for the answers. I prepurchased one now out of FOMO, though I'm still trying to figure out if I need to cancel the order or if I can trust that I'll actually be able to run stuff on it. Regarding LLMs, am I going to be able to run quantized models? Doesn't have to support all quantization methods, but at least one of the popular quantization methods would need to work (e.g. GGUF or EXL2). I ask this because quantized models provide the most "bang for buck" in LLMs. Regarding image generation, in real workflows there is typically a lot of dependencies (much more than in LLMs, and much more than in toy examples for image generation). I'm wondering how many of these dependencies will in practice be limited to NVIDIA and CUDA. For example, if you grab some popular ComfyUI Flux workflow from Reddit, are you able to actually run that on Framework Desktop?