Interesting. I saw some discussion on llama.cpp that, theoretically, implementing matmul for each quantization should be much faster since it can skip the conv

Interesting. I saw some discussion on llama.cpp that, theoretically, implementing matmul for each quantization should be much faster since it can skip the conversion. But practically, its actually quite difficult since the various BLAS libraries are so good.