2 ms·
Thanks! https://huggingface.co/lmsys/vicuna-13b-delta-v0 https://huggingface.co/lmsys/vicuna-13b-delta-v0 Edit, later: I found some instructive pages on how to
by superkuh 4y ago
Thanks! https://huggingface.co/lmsys/vicuna-13b-delta-v0 https://huggingface.co/lmsys/vicuna-13b-delta-v0
Edit, later: I found some instructive pages on how to use the vicuna weights with llama.cpp (https://lmsysvicuna.miraheze.org/wiki/How_to_use_Vicuna#Use_with_llama.cpp%3A https://lmsysvicuna.miraheze.org/wiki/How_to_use_Vicuna#Use_...) and pre-made ggml format compatible 4-bit quantized vicuna weights,
https://huggingface.co/eachadea/ggml-vicuna-13b-4bit/tree/main https://huggingface.co/eachadea/ggml-vicuna-13b-4bit/tree/ma... (8GB ready to go, no 60+GB RAM steps needed)