Depends on the model you want to run! That is the beauty of it. If you can muster that ability to run a Q4 Llama3.1 8B - great, if your specs are really low end

Depends on the model you want to run! That is the beauty of it. If you can muster that ability to run a Q4 Llama3.1 8B - great, if your specs are really low end you can always outsource to a cloud provider. Maybe you want to run privately, but can only manage a Gemma-2B - then great, you can use that also.