4 ms·
Out of curiosity, what degree of quantization are you applying to these 70B models?
by creata 2y ago
Out of curiosity, what degree of quantization are you applying to these 70B models?
- neop1x 2y agoQ4_K_S. While not as good as top commercial models like chatgpt, they are still quite capable and I like that there are also uncensored/abliterated models like Dolphin.