2 ms·It is Llama-2-70b-chat. I quantized it to 2q_k using `quantize` with llama.cpp.by beacon294 3y agoIt is Llama-2-70b-chat. I quantized it to 2q_k using `quantize` with llama.cpp.skavi 3y agoSo your experience isn’t representative of the work presented by this post? Or does llama.cpp use the same technique for quantization?beacon294 3y agoI don't know, hopefully it will help set general expectations.