> Is it an average between 2 and 3 over all weights? Yes I think it's an average where different quantization levels are used for different layers or weights.

> Is it an average between 2 and 3 over all weights? Yes I think it's an average where different quantization levels are used for different layers or weights. Here are more details about the quantization scheme: https://github.com/turboderp/exllamav2#exl2-quantization https://github.com/turboderp/exllamav2#exl2-quantization