4 ms·

I have 8GB RAM. So I am able to run 7B 4bit at maximum. I used GGUF + llama.cpp. Will try out Ollama. Thanks for the info!