3 ms·
It’s a shame that the article didn’t mention AWQ 4-bit quantization, which is quite widely supported in libraries and deployment tools (e.g. vLLM).
by woodson 2y ago
It’s a shame that the article didn’t mention AWQ 4-bit quantization, which is quite widely supported in libraries and deployment tools (e.g. vLLM).