2 ms·

btw I got 4bit quantized Vicuna working in my 16GB laptop and the results seem very good, perhaps the best I got running locally so far

• andai 4y ago
  Did you have to diff LLaMA? Did you use EasyLM?
  • muyuu 4y ago
    I found it ready-made for download, here https://huggingface.co/eachadea/ggml-vicuna-13b-4bit https://huggingface.co/eachadea/ggml-vicuna-13b-4bit