3 ms·
It has been quantized to 80GB (2-bit quantization for experts) with limited degradation. Certainly competitive with a 27B model, and especially useful in a siz
by zozbot234 4mo ago
It has been quantized to 80GB (2-bit quantization for experts) with limited degradation. Certainly competitive with a 27B model, and especially useful in a size range where few "native" models exist.
- lostmsu 4mo ago> (2-bit quantization for experts) with limited degradation. Certainly competitive with a 27B model Uh-huh...