4 ms·
Is there documentation somewhere on what the different quantisation methods are actually doing? E.g. what is Q4_0 vs Q4_1?
by sebzim4500 4y ago
Is there documentation somewhere on what the different quantisation methods are actually doing? E.g. what is Q4_0 vs Q4_1?
- ac29 4y agoThere's some info here: https://github.com/ggerganov/llama.cpp/issues/1240 https://github.com/ggerganov/llama.cpp/issues/1240