I’ve been using llama.cpp to try out some large-ish models on my cpu for a while now. I’m on an amd 3600, with 32gb ram. I can run 30B 4bit and 5bit models. It

I’ve been using llama.cpp to try out some large-ish models on my cpu for a while now. I’m on an amd 3600, with 32gb ram. I can run 30B 4bit and 5bit models. It’s definitely “usable” but smaller models definitely run faster. I don’t have a good graphics card, but it does work on cpu. With more ram I think the 65B and larger models are runnable as well