Better quantization will have a direct and meaningful impact for everyone running local LLMs. The technique has already been applied to both Vicuna and the mult

Better quantization will have a direct and meaningful impact for everyone running local LLMs. The technique has already been applied to both Vicuna and the multimodal LLaMA variant LLaVA. https://github.com/mit-han-lab/llm-awq https://github.com/mit-han-lab/llm-awq