3 ms·
> 1. Download the weights for the model you want to use, e.g. gpt4-x-vicuna-13B.ggml.q5_1.bin I think you need to quantize the model yourself from the float/hu
by rain1 3y ago
> 1. Download the weights for the model you want to use, e.g. gpt4-x-vicuna-13B.ggml.q5_1.bin
I think you need to quantize the model yourself from the float/huggingface versions. My understanding is that the quantization formats have changed recently. and old quantized models no longer work.
- rahimnathwani 3y agoThat was true until 2 days ago :) The repo has now been updated with requantized models that work with the latest version, so you don't need to do that any more. https://huggingface.co/TheBloke/gpt4-x-vicuna-13B-GGML/commit/b4b5f7e523f35306412d10ea9c4922b6f5923719 https://huggingface.co/TheBloke/gpt4-x-vicuna-13B-GGML/commi...
- rain1 3y agowonderful! thank you