4 ms·
ollama run mixtral will default to the quantized version (4bit IIRC). I'd guess this is why it can fit with two 3090s.
by _ea1k 3y ago
ollama run mixtral will default to the quantized version (4bit IIRC). I'd guess this is why it can fit with two 3090s.