5 ms·I wonder, can you quantize it yourself with some tool?by wkat4242 3y agoI wonder, can you quantize it yourself with some tool?pja 3y agollama.cpp can quantize a model for you: https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#quantization https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#qu...wkat4242 3y agoThanks!!