For people without 768gb+ of RAM and 64 cores+, this guide helps you set up systems to run the R1 distills, not the actual DeepSeek model itself which requires

For people without 768gb+ of RAM and 64 cores+, this guide helps you set up systems to run the R1 distills, not the actual DeepSeek model itself which requires a pretty beefy machine to run. (It is possible to built an AMD epic based machine that can put out around 4 t/s on a reasonable q4 of DeepSeek R1 for <$3000, with CPU inference) The distills offer a similar type of functionality implemented in other models such as Quen and LLAMA3… but are emphatically -not- DSR1, which is a very efficient and powerful SOTA model in it’s own right, even without the R1 reasoning fine tuning. To be clear, for most people, this guide is not about running DeepSeek R1 , but rather about running DeepSeek trained distills of other popular models.