AFAIK, 2-bit quant leads to too much loss of performance, such that you're better off using a different smaller model altogether. See here: https://www.reddit.

AFAIK, 2-bit quant leads to too much loss of performance, such that you're better off using a different smaller model altogether. See here: https://www.reddit.com/r/LocalLLaMA/comments/18ituzh/mixtral_update_on_perplexity_testing_adding_an/ https://www.reddit.com/r/LocalLLaMA/comments/18ituzh/mixtral...