3 ms·
I have dabbled with the quant version (exl), still not the same. Which quant/version are you using? There are so many that the output shifts dramatically!
by 3abiton 3y ago
I have dabbled with the quant version (exl), still not the same. Which quant/version are you using? There are so many that the output shifts dramatically!
- fzzzy 3y agoI'm using gguf q5 and I find the output degrades going to even q4. Mistral has said they still intend to open newer bigger models, so I'm really looking forward to that.