3 ms·
Looks like an issue with the quantization that ollama (i.e llama.cpp) uses and not the model itself. It's common knowledge from Mixtral 8x7B that quantizing the
by woadwarrior01 3y ago
Looks like an issue with the quantization that ollama (i.e llama.cpp) uses and not the model itself. It's common knowledge from Mixtral 8x7B that quantizing the MoE gates is pernicious to model perplexity. And yet they continue to do it. :)
- deleted 3y ago[deleted]
- cjbprime 3y agoNo, it's unrelated to quantization, they just weren't using the instruct model.