3 ms·
Do you have instructions and RAM requirements for running that model? Llama.cpp?
by pletnes 3y ago
Do you have instructions and RAM requirements for running that model? Llama.cpp?
- Casteil 3y agoI just use Ollama[1] - makes it incredibly easy to get going on MacOS, and can be run on linux/WSL also. RAM required will depend, but generally to run Mixtral at reasonable quantization levels (e.g. Q4) you're going to want 36GB or more. [1] https://github.com/jmorganca/ollama https://github.com/jmorganca/ollama