atharv_jaju

3 ms·

1. 1.▲
   by atharv_jaju 3y ago

   I have 8GB RAM. So I am able to run 7B 4bit at maximum. I used GGUF + llama.cpp. Will try out Ollama. Thanks for the info!
2. 2.▲
   by atharv_jaju 3y ago

   Oh my bad, I meant how to code it?
3. 3.▲
   by atharv_jaju 3y ago

   What is the best performing Codellama model on a Macbook with M1 chip? Will Gguf 7Bn 4bit quantized model be good to run locally on the Mac?
4. 4.▲
   by atharv_jaju 3y ago

   How do you count tokens per second?