For reference: I run 7B 5bit quantized models on a ryzen 7 5700G with 64Gb at 8 tokens/second CPU only. It's not close to what you can get with a high end graph

For reference: I run 7B 5bit quantized models on a ryzen 7 5700G with 64Gb at 8 tokens/second CPU only. It's not close to what you can get with a high end graphics card but for every day use it is alright and has headroom for bigger models. Upgrading CPU, RAM and Mainboard in a 10+year old PC cost me just 400€