4 ms·
I have two 3090s and it runs fine with `ollama run mixtral`. Although OP definitely meant mistral with the 7B note
by kwerk 3y ago
I have two 3090s and it runs fine with `ollama run mixtral`. Although OP definitely meant mistral with the 7B note
- _ea1k 3y agoollama run mixtral will default to the quantized version (4bit IIRC). I'd guess this is why it can fit with two 3090s.