3 ms·
Mixtral 8x7B. Short of that (high ram requirements), I've also found Mistral v0.2 to be pretty solid for a 7b model. YMMV though - it's going to depend on your
by Casteil 3y ago
Mixtral 8x7B. Short of that (high ram requirements), I've also found Mistral v0.2 to be pretty solid for a 7b model.
YMMV though - it's going to depend on your use cases.
- pletnes 3y agoDo you have instructions and RAM requirements for running that model? Llama.cpp?
- Casteil 3y agoI just use Ollama[1] - makes it incredibly easy to get going on MacOS, and can be run on linux/WSL also. RAM required will depend, but generally to run Mixtral at reasonable quantization levels (e.g. Q4) you're going to want 36GB or more. [1] https://github.com/jmorganca/ollama https://github.com/jmorganca/ollama