4 ms·
The official Q4_K_S gguf is quite good and has very good 35 tps generation on a M1 mac studio. Should be much faster on recent Macs, especially M5.
by tarruda 4mo ago
The official Q4_K_S gguf is quite good and has very good 35 tps generation on a M1 mac studio. Should be much faster on recent Macs, especially M5.
- SilverElfin 4mo agoWhat’s “Q4_K_S gguf” and where do I get it? Is it easy to install and configure on a MacBook?
- throw1234567891 4mo agohttps://huggingface.co/stepfun-ai/Step-3.7-Flash-GGUF https://huggingface.co/stepfun-ai/Step-3.7-Flash-GGUF and you can use ollama: https://docs.ollama.com/import#Importing-a-GGUF-based-model-or-adapter https://docs.ollama.com/import#Importing-a-GGUF-based-model-...